-- Logs begin at Wed 2022-08-31 18:31:49 UTC, end at Wed 2022-08-31 19:39:22 UTC. -- Aug 31 19:02:47.305747 dv-cinder-852210-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 19:02:47.313443 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 19:02:47 2022] *** Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: nodename: dv-cinder-852210-6-iscsi Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: machine: x86_64 Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: clock source: unix Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: pcre jit disabled Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: detected number of CPU cores: 4 Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: current working directory: / Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: detected binary path: /usr/bin/uwsgi-core Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: your processes number limit is 47680 Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: your memory page size is 4096 bytes Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: detected max file descriptor number: 2048 Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: lock engine: pthread robust mutexes Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: thunder lock: enabled Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 19:02:47.327683 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 19:02:47.361774 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: Python main interpreter initialized at 0x559c6f9f09c0 Aug 31 19:02:47.361774 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: python threads support enabled Aug 31 19:02:47.361774 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: your server socket listen backlog is limited to 100 connections Aug 31 19:02:47.361774 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: your mercy for graceful operations on workers is 90 seconds Aug 31 19:02:47.361774 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 19:02:47.361774 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: *** Operational MODE: preforking *** Aug 31 19:02:47.362254 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: *** uWSGI is running in multiple interpreter mode *** Aug 31 19:02:47.362254 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: spawned uWSGI master process (pid: 81576) Aug 31 19:02:47.362146 dv-cinder-852210-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 19:02:47.368618 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: spawned uWSGI worker 1 (pid: 81591, cores: 1) Aug 31 19:02:47.368618 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: spawned uWSGI worker 2 (pid: 81592, cores: 1) Aug 31 19:02:47.368618 dv-cinder-852210-6-iscsi devstack@keystone.service[81576]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 19:02:52.299637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 19:02:52.299637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: from cryptography.utils import int_from_bytes Aug 31 19:02:52.301067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 19:02:52.301067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: from cryptography.utils import int_from_bytes Aug 31 19:02:52.351582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 19:02:52.351785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 19:02:52.351977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] command line args: [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 19:02:52.352241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 19:02:52.352420 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ================================================================================ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 19:02:52.352707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] admin_token = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.352958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] config_dir = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.353150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.353335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] config_source = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.353519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] debug = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.353721 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.354986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.354986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.354986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.355294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.355294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.355503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_config_append = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.355614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.355731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_dir = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.355971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_file = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.356152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.356402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.356676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.356927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.357162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.357345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.357596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.358489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.358747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.358989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.359225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.359459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.359709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.359958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.360233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.360749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.95/identity {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.360937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] publish_errors = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.361175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.361295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 19:02:52.361295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: from cryptography.utils import int_from_bytes Aug 31 19:02:52.361979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.361979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.362194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 19:02:52.362194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: from cryptography.utils import int_from_bytes Aug 31 19:02:52.362400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.362400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.362400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] standard_threads = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.362722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.362722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.362923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.363042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_journal = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.363195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_json = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.363373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_stderr = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.363551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_syslog = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.363783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.364042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.364282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.364541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.364807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.365059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.365300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.365549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.365789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.366038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.366285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.366534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.366770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.367025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.367268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.367514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.367762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.368002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.368256 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.368510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.368787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.369026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.369282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.369546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.369800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.370049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.370283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.370516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.370749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.370986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.371237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.371485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.371718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.371951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.372198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.372444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.372708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.372959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.373194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.373485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.373739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.374156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.374467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.374708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.374946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.375192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.375445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.375680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.375919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.376150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.376390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.376655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.376911 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.377152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.377433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.377631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.377870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.378110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.378351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.378586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.378840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.379081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.379324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.379562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.379827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.380067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.380300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.380537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.380804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.381078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.381317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.381566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.381814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.382055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.382294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.382534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.382782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.383021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.383260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.383504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.383742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.383992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.384234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.384473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.384737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.384979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.385220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.385483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.385727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.385975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.386214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.386454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.386695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.386939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.387175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.387413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.387647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.387878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.388109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.388343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.388593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.388843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.389108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.389446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.389841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.390275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.390554 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.390805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.391045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.external = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.391282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.391584 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.391821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.392055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.password = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.392290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.token = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.392536 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.392802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.393056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.393294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.393545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.393797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.394034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.394274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.394527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.394781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.395041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.395288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.395535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.395771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.396020 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.396276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.396516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.396805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.397087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.397339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.397604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.397848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.398107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.398346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.398586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.398834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.399081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.399315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.399565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.399804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.400059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.400299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.400555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.400818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.401069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.401307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.401558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.401807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.402045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.402282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.402530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.402765 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.403008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.403259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.403513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.403747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.403978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.404212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.404459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.405340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.405718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.406154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.406415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.406695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.406945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.407187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.407424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.407664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.407897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.408138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.408380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.408646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.408900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.409134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.409384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.409636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.409873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.410122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.410291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 19:02:52.410404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.410513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 19:02:52.410645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.410764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] command line args: [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 19:02:52.410882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.410994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 19:02:52.411108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.411226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ================================================================================ {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 19:02:52.411344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.411467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] admin_token = **** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.411578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.411705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] config_dir = [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.411823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.411936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.411936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] config_source = [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.412146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.412255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] debug = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.412368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.412480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.412698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.412824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.412824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.413031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.413031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.413249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.413357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.413486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.413609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.413721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] list_limit = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.413721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] log_config_append = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.413931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.414043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.414173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.414282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] log_dir = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.414282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] log_file = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.414491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.414603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.414716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.414840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.414950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.415070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.415185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.415292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.415426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.415537 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.415651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.415774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.415886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.415998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.416123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.416233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.416348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.416465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.416612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.416748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.416868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.416990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.417098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.417221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.417332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.417446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.417559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.417683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.417793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.417907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.418020 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.418020 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.418236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.95/identity {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.418362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.418472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] publish_errors = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.418585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.418698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.418821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.418933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.418933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.419462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.419579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.419781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.419906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.420021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.420134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] standard_threads = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.420866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.421020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.421140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.421252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.421362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.421487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.421599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.421713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] use_journal = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.421713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] use_json = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.421931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.422166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] use_stderr = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.422294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.422419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] use_syslog = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.422533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.422648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 19:02:52.422648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.422863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.422988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.423107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.423221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.423336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.423451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.423451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.423681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.423791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.423791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.424918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.425048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.425159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.425275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.425275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.425737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.425847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.425847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.426074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.426197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.426312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.426427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.426427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.426654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.426765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.426880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.427927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.428055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.428170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.428288 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.428412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.428538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.428707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] role.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.428825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.428943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] role.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.430107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.430218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.430367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.430479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.430614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.430732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.430855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.430966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.431082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.431196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.431310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.431424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.431621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.431730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.431852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.431964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.432080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.432199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.432324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.432437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.432547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.432697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.432814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.432931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.433066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.433178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.433298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.433414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.433533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.433649 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.433778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.433898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434362 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.434842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.435052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.435187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.435300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.435300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.435535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.435649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.436299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.436299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.436507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.436683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.436807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.436919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.437100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.437235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.437405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.437514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.437635 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.437748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.437862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.437996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.438105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.438218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.438333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.438449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.438586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.438709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.438820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.438933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439644 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.439985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.440113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.440224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.440339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.440453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.440592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.440716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.440852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.440969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.441088 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.441204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.441323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.441458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.441571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.441689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.441806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.441924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.442057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.442183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.442299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.442427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.442546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.442663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.442780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.442897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.443970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.444087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.444208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.444465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.444632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.444763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.444873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.444989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.445103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.445103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.445318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.445442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.445554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.445667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.445780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.445895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.446963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.447078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.447195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] auth.external = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.447311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.447426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.447558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 19:02:52.447669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.447859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.448092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] auth.password = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.448325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] auth.token = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.448562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.448818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.449066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.449298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.449545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.449793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.450024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.450259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] credential.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.450506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.450752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.450997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.451254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.452183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.452444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.452885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.453147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.453388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.453641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.453925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.454172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.454433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.454675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.454928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.455163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] federation.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.455399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.455643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.455886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.456115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.456364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.456619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.456872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.457108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.457360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.457598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.457844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.458077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.458323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.458566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.458799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.459033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.459277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.459506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.459743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.459991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.460241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.460471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.460721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.460952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.461195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.461441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.461685 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.461936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.462180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.462449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.462686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.462924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.463156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.463391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.463621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.463855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.464095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.464335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.464598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.464830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.465073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.465316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.465549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.465794 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.466038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.466277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.466512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.466741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.466979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.467466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.467769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.468200 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.468746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.469147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.469602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.469947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.470301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.470641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.471000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.471350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.471759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.472011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.472251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.472501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.472789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.473026 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.473274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.473525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.473775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.474012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.474244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.474481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.474716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.474956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.475186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.475433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.475679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.475925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.476169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.476420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.476671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.476920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.477158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.477394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.477633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.477867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.478113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.478361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.478600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.478748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.478885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.479013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.479139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.479270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.479398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.479541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.479751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.479989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.480233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.480474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.480730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.480965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.481205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] resource.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.481455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.481703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.481934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.482181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.482430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.482813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.483039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.483180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.483420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.483656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] role.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.483903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] role.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.484130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.484379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.484645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.484895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.485140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.485389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.485637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.485873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.486119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.486358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.486598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.486842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.487087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.487333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.487560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.487757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.487994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.488244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.488494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.489008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.489242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.489496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.489749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.489988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.490221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.490452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.490681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.490931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.491187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.491433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.491669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.491905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.492135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.492370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] token.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.492621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.492867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.493103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.493357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.493603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.493842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.494087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.494327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.494574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.494809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.495049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.495280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.495521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.495802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.496029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.496276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.496541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.496820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.connection = **** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.497065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.497308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.497546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.497789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.498029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.498789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.499097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.499743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.499743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.499967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.500097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.500250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.500498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.500757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.501004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.501233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.501472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.501706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 19:02:52.501943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 19:02:52.516859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.517102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.527293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.527598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.527895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.537061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.556100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.556324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.572237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.596057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.622451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.622691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.631941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.632113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.781652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.781948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.782083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.782298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.782517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.782737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.783646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.783881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.784107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.784328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.784545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.784783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.806439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.806675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.806875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.806967 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.807986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.808201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.819509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.819655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.833474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.833686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.833686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.833926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.834074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.834199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.834324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.834446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.834569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.834693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.834828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.834952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.835071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81592) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.835190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 19:02:52.852413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.852632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.852770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 19:02:52.852899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 19:02:52.853026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.853151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.853273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 19:02:52.853417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 19:02:52.853532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.853659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.853779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.853958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.854716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.854832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.854951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.854951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.863801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.864085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.864291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.864378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.864598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.864685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.864869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.865156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.871648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.872106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.874851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.875075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.875177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.875364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.875447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.875650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.875732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.875933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.876026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.876197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.876278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.876466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.876546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.876763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.876848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.877039 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.877150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.877318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.877412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.877595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.877696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.877970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.901041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.901194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.902453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.902579 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.902745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.902870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.907867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.907995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.908157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.908246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.908445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.908531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.914800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.914949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.915090 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.915237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.919208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.919374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.919534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.919662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.919787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.919914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.920035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.920208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.920346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.920645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.920730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.920863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.921321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.922425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.922723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.923013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.939392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.939508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.939667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.939755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.943541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.943648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.946797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.946904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.947095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.947177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.951327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.951504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.955069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.955221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.958312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.958410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.958612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.958709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.958901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.959117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.959198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.959410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.959500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.959706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.959788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.960098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.975469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.975759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.976028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.976163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.976300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.976454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.976823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.977106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.983343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.983980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.984919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.985697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.986463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.986741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.987241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.987521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.989394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.989678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.990080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.990221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:52.990375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.990512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.990664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.990809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.991108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:52.991396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.002455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.002740 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.003119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.003407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.003608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.003705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.003890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.003988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.004174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.004271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.004458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.004770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.005053 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.005334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.020143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.021488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.023077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.024529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.025964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.027400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.027485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.028962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.029089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.029340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.029651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.031394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.031783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.032073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.036063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.037572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.038610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.040112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.040557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.040871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.041161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.041441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.043033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.043329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.043629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.043915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.049520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.049812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.050093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.052325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.052651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.052939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.057283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81592) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.057653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.058017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.058383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.058738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.059091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.059450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.059815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.059969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 19:02:53.060126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.060266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.060423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.060688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.061403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.061548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.061814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.062189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.062335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.062491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.062632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.062769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.062910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.063142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.063304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81592) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.063442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.063670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.063964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.064230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 19:02:53.100219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559c6f9f09c0 pid: 81592 (default app) Aug 31 19:02:53.103598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3065-ec78-47b5-ad2e-ddf4032c155a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:53.103888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3065-ec78-47b5-ad2e-ddf4032c155a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:53.104161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d3065-ec78-47b5-ad2e-ddf4032c155a None None] PATH_INFO: `/v3` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:53.107482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559c6f9f09c0 pid: 81591 (default app) Aug 31 19:02:53.109508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1/1] 10.222.0.95 () {50 vars in 791 bytes} [Wed Aug 31 19:02:53 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:59.801489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126346-9e0c-41a4-9516-9923d96b7318 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:59.802019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126346-9e0c-41a4-9516-9923d96b7318 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:59.802019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126346-9e0c-41a4-9516-9923d96b7318 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:59.806279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1/2] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:02:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:02:59.813043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:59.813671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:59.813671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:59.885476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: INFO dbcounter [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] Registered counter for database keystone Aug 31 19:02:59.904785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG oslo_db.sqlalchemy.engines [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81592) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 19:02:59.911443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writer thread running {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 19:03:00.098280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG passlib.handlers.bcrypt [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81592) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 19:03:00.098588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG passlib.handlers.bcrypt [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81592) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 19:03:00.159400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:00.161158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:00.309827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 2/3] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:02:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 19:03:00.777436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb126346-9e0c-41a4-9516-9923d96b7318 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:00.850918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: INFO dbcounter [None req-bb126346-9e0c-41a4-9516-9923d96b7318 None None] Registered counter for database keystone Aug 31 19:03:00.875087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG oslo_db.sqlalchemy.engines [None req-bb126346-9e0c-41a4-9516-9923d96b7318 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81591) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 19:03:00.885530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writer thread running {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 19:03:01.009498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb126346-9e0c-41a4-9516-9923d96b7318 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:01.014191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a69ee-a36d-4bf0-bff2-b5a9e71cb7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:01.014839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a69ee-a36d-4bf0-bff2-b5a9e71cb7fa admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:01.015126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a69ee-a36d-4bf0-bff2-b5a9e71cb7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:01.015399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a69ee-a36d-4bf0-bff2-b5a9e71cb7fa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:01.024010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a69ee-a36d-4bf0-bff2-b5a9e71cb7fa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:01.078820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7b4a69ee-a36d-4bf0-bff2-b5a9e71cb7fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 19:03:01.079084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a69ee-a36d-4bf0-bff2-b5a9e71cb7fa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:01.089826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 2/4] 10.222.0.95 () {58 vars in 1324 bytes} [Wed Aug 31 19:03:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:03:01.096644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:01.112907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b278a8e-9fdf-445d-ab4e-de48f2f77ed7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:01.117724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27787cd3-401b-48ea-9c2d-572f7e89fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:01.118845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27787cd3-401b-48ea-9c2d-572f7e89fa90 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:01.119124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27787cd3-401b-48ea-9c2d-572f7e89fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:01.119691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27787cd3-401b-48ea-9c2d-572f7e89fa90 admin admin] PATH_INFO: `/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:01.121284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27787cd3-401b-48ea-9c2d-572f7e89fa90 admin admin] RBAC: Authorizing `identity:get_project(project_id=4b2aa0a8faf9471ebd699445da59c0dc)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:01.178542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING keystone.common.rbac_enforcer.enforcer [None req-27787cd3-401b-48ea-9c2d-572f7e89fa90 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 19:03:01.178880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27787cd3-401b-48ea-9c2d-572f7e89fa90 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:01.183611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 3/5] 10.222.0.95 () {56 vars in 1207 bytes} [Wed Aug 31 19:03:01 2022] GET /identity/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:03.108177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b714dca-f0da-40f8-94f3-5980fd94c1c4 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:03.108787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b714dca-f0da-40f8-94f3-5980fd94c1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:03.108932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b714dca-f0da-40f8-94f3-5980fd94c1c4 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:03.113050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 3/6] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:03.118556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d1e2a-2d93-4650-a9bc-3a200abae3d7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:03.118972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d1e2a-2d93-4650-a9bc-3a200abae3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:03.119080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d1e2a-2d93-4650-a9bc-3a200abae3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:03.139496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-bb0d1e2a-2d93-4650-a9bc-3a200abae3d7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:03.141077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-bb0d1e2a-2d93-4650-a9bc-3a200abae3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:03.149038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 4/7] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 19:03:03.418699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b714dca-f0da-40f8-94f3-5980fd94c1c4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:03.431776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b714dca-f0da-40f8-94f3-5980fd94c1c4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:03.436711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eebd6d-f9fb-4e23-a59a-cc59b0290ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:03.437309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eebd6d-f9fb-4e23-a59a-cc59b0290ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:03.437593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eebd6d-f9fb-4e23-a59a-cc59b0290ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:03.437867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eebd6d-f9fb-4e23-a59a-cc59b0290ddf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:03.445603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eebd6d-f9fb-4e23-a59a-cc59b0290ddf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:03.447377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eebd6d-f9fb-4e23-a59a-cc59b0290ddf admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:03.456039 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 4/8] 10.222.0.95 () {58 vars in 1324 bytes} [Wed Aug 31 19:03:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:03:03.461644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d1e2a-2d93-4650-a9bc-3a200abae3d7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:03.471536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d1e2a-2d93-4650-a9bc-3a200abae3d7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:03.475919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb3974-74cb-4266-847d-0e610087a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:03.476544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb3974-74cb-4266-847d-0e610087a243 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:03.476857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb3974-74cb-4266-847d-0e610087a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:03.477136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb3974-74cb-4266-847d-0e610087a243 admin admin] PATH_INFO: `/v3/users/60fc2717779e40a0a55137c62461f753` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:03.478473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb3974-74cb-4266-847d-0e610087a243 admin admin] RBAC: Authorizing `identity:get_user(user_id=60fc2717779e40a0a55137c62461f753)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:03.480322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb3974-74cb-4266-847d-0e610087a243 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:03.481902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 5/9] 10.222.0.95 () {56 vars in 1198 bytes} [Wed Aug 31 19:03:03 2022] GET /identity/v3/users/60fc2717779e40a0a55137c62461f753 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:05.500390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d56224-b286-40bb-9922-ee4278706b1a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:05.500972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d56224-b286-40bb-9922-ee4278706b1a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:05.500972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d56224-b286-40bb-9922-ee4278706b1a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:05.502276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 5/10] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:05.507482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4613b3-140e-446b-b6af-69c091d8e945 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:05.507921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4613b3-140e-446b-b6af-69c091d8e945 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:05.508112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4613b3-140e-446b-b6af-69c091d8e945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:05.526888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-ba4613b3-140e-446b-b6af-69c091d8e945 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:05.528096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ba4613b3-140e-446b-b6af-69c091d8e945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:05.534590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0b3e-2fb4-4d62-a98c-6171215dafa9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:05.534920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0b3e-2fb4-4d62-a98c-6171215dafa9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:05.535189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c0b3e-2fb4-4d62-a98c-6171215dafa9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:05.536511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 6/11] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:03:05.541552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:05.541843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:05.542121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:05.585994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG passlib.handlers.bcrypt [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81591) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 19:03:05.586395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG passlib.handlers.bcrypt [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81591) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 19:03:05.611516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 6/12] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:05.635891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:05.637552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:05.742111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 7/13] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:05.880395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4613b3-140e-446b-b6af-69c091d8e945 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:05.905257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:05.989569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4613b3-140e-446b-b6af-69c091d8e945 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.015892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.024371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d981e1-920c-4f68-abd5-6b6ec5acb8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.025034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d981e1-920c-4f68-abd5-6b6ec5acb8da None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.025314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d981e1-920c-4f68-abd5-6b6ec5acb8da None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.025586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d981e1-920c-4f68-abd5-6b6ec5acb8da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.037637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d981e1-920c-4f68-abd5-6b6ec5acb8da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-d0d981e1-920c-4f68-abd5-6b6ec5acb8da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:06.046633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:06.047714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:06.048752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 7/14] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:06.053770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d981e1-920c-4f68-abd5-6b6ec5acb8da None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.125729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e522bad-18a5-4b7c-82b3-de019e78c1a5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.128155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:06.141147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:06.161312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d981e1-920c-4f68-abd5-6b6ec5acb8da None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.167215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9622ec7-2f1b-414f-a5d2-33b8e07ca039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.168076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9622ec7-2f1b-414f-a5d2-33b8e07ca039 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.168355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9622ec7-2f1b-414f-a5d2-33b8e07ca039 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.168653 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9622ec7-2f1b-414f-a5d2-33b8e07ca039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.195319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.195920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.196199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.196464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.197205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:06.197506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:06.198895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:06.210596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 8/15] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:06.216291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.238848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9622ec7-2f1b-414f-a5d2-33b8e07ca039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:06.240408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9622ec7-2f1b-414f-a5d2-33b8e07ca039 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:06.319985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7e9be-3b21-488a-9066-f7e3773c3dd6 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.352848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 8/16] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:03:06.355087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1728597-d955-4430-a735-f332c2c94d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.355759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728597-d955-4430-a735-f332c2c94d13 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.356042 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728597-d955-4430-a735-f332c2c94d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.356314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1728597-d955-4430-a735-f332c2c94d13 None admin] PATH_INFO: `/v3/users/60fc2717779e40a0a55137c62461f753` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.358854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1728597-d955-4430-a735-f332c2c94d13 None admin] RBAC: Authorizing `identity:get_user(user_id=60fc2717779e40a0a55137c62461f753)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:06.359182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9622ec7-2f1b-414f-a5d2-33b8e07ca039 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.360671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1728597-d955-4430-a735-f332c2c94d13 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:06.362618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:06.362736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 9/17] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/users/60fc2717779e40a0a55137c62461f753 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:06.368753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1728597-d955-4430-a735-f332c2c94d13 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.383977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:06.475477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9622ec7-2f1b-414f-a5d2-33b8e07ca039 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.485837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1728597-d955-4430-a735-f332c2c94d13 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.510450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.511312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.511594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.511869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.522295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2e10a-cd48-4ad6-9be9-72d1307bcb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.522929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2e10a-cd48-4ad6-9be9-72d1307bcb65 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.523208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2e10a-cd48-4ad6-9be9-72d1307bcb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.523482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2e10a-cd48-4ad6-9be9-72d1307bcb65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.524630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2e10a-cd48-4ad6-9be9-72d1307bcb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:06.526114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2e10a-cd48-4ad6-9be9-72d1307bcb65 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:06.527655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 10/18] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:06.533109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:06.534326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2e10a-cd48-4ad6-9be9-72d1307bcb65 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.534614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:03:06.553639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:06.554741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:03:06.555549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:03:06.555549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:03:06.555549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:06.555549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 9/19] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:06.560867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.620309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:06.630627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:06.641861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2e10a-cd48-4ad6-9be9-72d1307bcb65 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.677316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61828a8-e1ed-400e-82de-c7b3d5e9b110 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.689987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d446b-e386-45dc-8ce5-3c1e64f23f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.690944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d446b-e386-45dc-8ce5-3c1e64f23f54 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.691406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d446b-e386-45dc-8ce5-3c1e64f23f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.691855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d446b-e386-45dc-8ce5-3c1e64f23f54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.693034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d446b-e386-45dc-8ce5-3c1e64f23f54 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.domain.id=default, user.id=60fc2717779e40a0a55137c62461f753) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:06.693545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d446b-e386-45dc-8ce5-3c1e64f23f54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:06.696065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d446b-e386-45dc-8ce5-3c1e64f23f54 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:06.706648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 11/20] 10.222.0.95 () {56 vars in 1322 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/role_assignments?user.id=60fc2717779e40a0a55137c62461f753&scope.domain.id=default&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 233 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:06.724757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc499c0-3e58-42eb-8c54-c6b6e3b72b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.725723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc499c0-3e58-42eb-8c54-c6b6e3b72b18 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.726085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc499c0-3e58-42eb-8c54-c6b6e3b72b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.726496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc499c0-3e58-42eb-8c54-c6b6e3b72b18 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.727486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc499c0-3e58-42eb-8c54-c6b6e3b72b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:06.727946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc499c0-3e58-42eb-8c54-c6b6e3b72b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:06.730221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc499c0-3e58-42eb-8c54-c6b6e3b72b18 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:06.760369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 10/21] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:08.962573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373b57b-3f7e-40f7-9179-26e622d943fe None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:08.963095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373b57b-3f7e-40f7-9179-26e622d943fe None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:08.963095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373b57b-3f7e-40f7-9179-26e622d943fe None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:08.964698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 12/22] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:08.971339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d07994-9562-4bc5-aec2-3708249ff2d9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:08.972300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d07994-9562-4bc5-aec2-3708249ff2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:08.972811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d07994-9562-4bc5-aec2-3708249ff2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:08.995261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-25d07994-9562-4bc5-aec2-3708249ff2d9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:08.996791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-25d07994-9562-4bc5-aec2-3708249ff2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:09.078009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 11/23] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:09.203179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b7174-c7a9-434e-8ed8-f511136a4618 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.203841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b7174-c7a9-434e-8ed8-f511136a4618 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.204324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b7174-c7a9-434e-8ed8-f511136a4618 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.206441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 13/24] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:09.212366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8cea-9242-4636-b144-67c03b2be528 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.213259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8cea-9242-4636-b144-67c03b2be528 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.213908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8cea-9242-4636-b144-67c03b2be528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.244112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1e9e8cea-9242-4636-b144-67c03b2be528 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:09.245770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1e9e8cea-9242-4636-b144-67c03b2be528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:09.286652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed21953-085e-4f81-a464-ddad9c8604e2 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.287335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed21953-085e-4f81-a464-ddad9c8604e2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.287827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed21953-085e-4f81-a464-ddad9c8604e2 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.289402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 14/25] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:03:09.296586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5a492-d0a9-4ce7-83f0-4419544a9eef None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.297269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5a492-d0a9-4ce7-83f0-4419544a9eef None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.297745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5a492-d0a9-4ce7-83f0-4419544a9eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.320203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-19f5a492-d0a9-4ce7-83f0-4419544a9eef None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:09.320920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:09.321979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-19f5a492-d0a9-4ce7-83f0-4419544a9eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:09.360178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 12/26] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:09.398199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 15/27] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:09.645054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8cea-9242-4636-b144-67c03b2be528 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:09.705757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5a492-d0a9-4ce7-83f0-4419544a9eef None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:09.761305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8cea-9242-4636-b144-67c03b2be528 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:09.799545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e094f1-578d-4834-8804-83b278dc5a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:09.800194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e094f1-578d-4834-8804-83b278dc5a20 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.800770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e094f1-578d-4834-8804-83b278dc5a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.801045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e094f1-578d-4834-8804-83b278dc5a20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.814632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e094f1-578d-4834-8804-83b278dc5a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a9e094f1-578d-4834-8804-83b278dc5a20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:09.821359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:09.822226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:09.823004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 13/28] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:09.829604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdccedf-993e-4411-b667-ac24fc2661d1 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.829919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdccedf-993e-4411-b667-ac24fc2661d1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.830149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdccedf-993e-4411-b667-ac24fc2661d1 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.831638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 14/29] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:03:09.832162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdccedf-993e-4411-b667-ac24fc2661d1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:09.880621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:09.885134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:09.916313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5a492-d0a9-4ce7-83f0-4419544a9eef None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:09.958949 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084720ae-a1f6-499b-9280-fd3ae3932fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:09.959591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084720ae-a1f6-499b-9280-fd3ae3932fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.959876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084720ae-a1f6-499b-9280-fd3ae3932fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.960150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084720ae-a1f6-499b-9280-fd3ae3932fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.971598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084720ae-a1f6-499b-9280-fd3ae3932fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:09.973167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084720ae-a1f6-499b-9280-fd3ae3932fb6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:09.978256 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdccedf-993e-4411-b667-ac24fc2661d1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-084720ae-a1f6-499b-9280-fd3ae3932fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:03:09.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:09.991452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:03:09.992325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:03:09.992325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:03:09.992325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:09.992325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 16/30] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:03:09.993145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc164a-7ced-48be-8c91-ebb86c3b0b04 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.993433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc164a-7ced-48be-8c91-ebb86c3b0b04 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.993766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc164a-7ced-48be-8c91-ebb86c3b0b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:10.015259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c2cc164a-7ced-48be-8c91-ebb86c3b0b04 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:10.016511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c2cc164a-7ced-48be-8c91-ebb86c3b0b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:10.024301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:10.025774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:10.026255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:10.026747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:10.028076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:10.028647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:10.031063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:10.052611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 15/31] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:10.052611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:10.111108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 17/32] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:10.112899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc164a-7ced-48be-8c91-ebb86c3b0b04 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:10.183722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:10.188074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed744f-daca-4af1-b309-cc32decc85bf None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:10.205672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:10.250905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e526cdb-3a20-4368-a7cf-c608a7c413e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:10.252246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e526cdb-3a20-4368-a7cf-c608a7c413e2 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:10.253015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e526cdb-3a20-4368-a7cf-c608a7c413e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:10.253638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e526cdb-3a20-4368-a7cf-c608a7c413e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:10.255150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e526cdb-3a20-4368-a7cf-c608a7c413e2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:10.258146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e526cdb-3a20-4368-a7cf-c608a7c413e2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:10.282449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc164a-7ced-48be-8c91-ebb86c3b0b04 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:10.328879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:10.329877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:10.330416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:10.330871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:10.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:10.332209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:10.338030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:10.374171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 18/33] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:10.375382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:10.463022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 16/34] 10.222.0.95 () {60 vars in 1131 bytes} [Wed Aug 31 19:03:10 2022] POST /identity/v3/roles => generated 219 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:10.464367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e526cdb-3a20-4368-a7cf-c608a7c413e2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:10.579130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efffa6-014a-4c86-845e-578b6c2631b5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:10.634390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:10.635493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ba88e-69b5-4f2b-950b-d93775824ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:10.636192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ba88e-69b5-4f2b-950b-d93775824ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:10.636767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ba88e-69b5-4f2b-950b-d93775824ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:10.637052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ba88e-69b5-4f2b-950b-d93775824ad4 None admin] PATH_INFO: `/v3/users/60fc2717779e40a0a55137c62461f753` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:10.640266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ba88e-69b5-4f2b-950b-d93775824ad4 None admin] RBAC: Authorizing `identity:get_user(user_id=60fc2717779e40a0a55137c62461f753)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:10.641807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ba88e-69b5-4f2b-950b-d93775824ad4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:10.643526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 19/35] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:10 2022] GET /identity/v3/users/60fc2717779e40a0a55137c62461f753 => generated 247 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:10.646755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ba88e-69b5-4f2b-950b-d93775824ad4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:10.738648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e526cdb-3a20-4368-a7cf-c608a7c413e2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:10.791755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027c74c-c1da-4116-a7c3-50f9cdc7e1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:10.792697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027c74c-c1da-4116-a7c3-50f9cdc7e1ad None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:10.792994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027c74c-c1da-4116-a7c3-50f9cdc7e1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:10.793272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027c74c-c1da-4116-a7c3-50f9cdc7e1ad None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:10.794213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027c74c-c1da-4116-a7c3-50f9cdc7e1ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:10.796856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027c74c-c1da-4116-a7c3-50f9cdc7e1ad None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:10.852758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 17/36] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:03:10 2022] POST /identity/v3/projects => generated 301 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:10.855000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027c74c-c1da-4116-a7c3-50f9cdc7e1ad None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:10.872058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ba88e-69b5-4f2b-950b-d93775824ad4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:10.879902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:10.940633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a999b44-a355-44ce-9bd6-a190ab82993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:10.948702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a999b44-a355-44ce-9bd6-a190ab82993c None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:10.948702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a999b44-a355-44ce-9bd6-a190ab82993c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:10.948702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a999b44-a355-44ce-9bd6-a190ab82993c None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:10.948702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a999b44-a355-44ce-9bd6-a190ab82993c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:10.948702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a999b44-a355-44ce-9bd6-a190ab82993c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:11.057172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027c74c-c1da-4116-a7c3-50f9cdc7e1ad None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:11.104321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbbce2-c2a1-4d6b-967b-44b5c86b13e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:11.105363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbbce2-c2a1-4d6b-967b-44b5c86b13e4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:11.105882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbbce2-c2a1-4d6b-967b-44b5c86b13e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:11.106364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbbce2-c2a1-4d6b-967b-44b5c86b13e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:11.109508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 20/37] 10.222.0.95 () {60 vars in 1131 bytes} [Wed Aug 31 19:03:10 2022] POST /identity/v3/roles => generated 225 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:11.110727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920a9a-748a-489c-9070-661435cf620d None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:11.111023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920a9a-748a-489c-9070-661435cf620d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:11.111207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbbce2-c2a1-4d6b-967b-44b5c86b13e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:11.111318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa920a9a-748a-489c-9070-661435cf620d None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:11.112645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 21/38] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:11.113315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbbce2-c2a1-4d6b-967b-44b5c86b13e4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:11.115196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 18/39] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:11.120723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87662c9-cbcb-47a2-92da-742adc434142 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:11.121513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87662c9-cbcb-47a2-92da-742adc434142 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:11.122053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87662c9-cbcb-47a2-92da-742adc434142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:11.124744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbbce2-c2a1-4d6b-967b-44b5c86b13e4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:11.149966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f87662c9-cbcb-47a2-92da-742adc434142 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:11.151565 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f87662c9-cbcb-47a2-92da-742adc434142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:11.211187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:11.260663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:11.276620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 22/40] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:11.276620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a075b-ff89-4ab6-afc3-22d8eb7681a2 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:11.276620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a075b-ff89-4ab6-afc3-22d8eb7681a2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:11.276620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a075b-ff89-4ab6-afc3-22d8eb7681a2 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:11.276620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 23/41] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:11.284188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5487f-c2b7-461a-93f3-4ad59b175a39 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:11.285493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5487f-c2b7-461a-93f3-4ad59b175a39 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:11.286617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5487f-c2b7-461a-93f3-4ad59b175a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:11.291875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbbce2-c2a1-4d6b-967b-44b5c86b13e4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:11.325517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4ee5487f-c2b7-461a-93f3-4ad59b175a39 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:11.327631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4ee5487f-c2b7-461a-93f3-4ad59b175a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:11.337473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac660c51-3f5d-413f-a1d9-f46055303595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:11.338386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac660c51-3f5d-413f-a1d9-f46055303595 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:11.338863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac660c51-3f5d-413f-a1d9-f46055303595 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:11.339308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac660c51-3f5d-413f-a1d9-f46055303595 None admin] PATH_INFO: `/v3/domains/default/users/60fc2717779e40a0a55137c62461f753/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:11.342373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac660c51-3f5d-413f-a1d9-f46055303595 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=60fc2717779e40a0a55137c62461f753, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:11.344246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac660c51-3f5d-413f-a1d9-f46055303595 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:11.370083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 19/42] 10.222.0.95 () {58 vars in 1361 bytes} [Wed Aug 31 19:03:11 2022] PUT /identity/v3/domains/default/users/60fc2717779e40a0a55137c62461f753/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:11.460049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 24/43] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:11.598336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac660c51-3f5d-413f-a1d9-f46055303595 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:11.706679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac660c51-3f5d-413f-a1d9-f46055303595 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:11.730198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5487f-c2b7-461a-93f3-4ad59b175a39 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:11.765692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56793607-d6e5-4789-a154-40a783ba2d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:11.766332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56793607-d6e5-4789-a154-40a783ba2d96 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:11.766619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56793607-d6e5-4789-a154-40a783ba2d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:11.766899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56793607-d6e5-4789-a154-40a783ba2d96 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:11.767606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56793607-d6e5-4789-a154-40a783ba2d96 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:11.769097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56793607-d6e5-4789-a154-40a783ba2d96 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:11.779970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:11.796128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 20/44] 10.222.0.95 () {60 vars in 1131 bytes} [Wed Aug 31 19:03:11 2022] POST /identity/v3/roles => generated 223 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:11.974375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5487f-c2b7-461a-93f3-4ad59b175a39 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:11.977826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:12.014364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd16dd1-1e71-42f6-a63b-4bbe3221f956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:12.015005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd16dd1-1e71-42f6-a63b-4bbe3221f956 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:12.015288 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd16dd1-1e71-42f6-a63b-4bbe3221f956 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:12.015559 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd16dd1-1e71-42f6-a63b-4bbe3221f956 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:12.016742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd16dd1-1e71-42f6-a63b-4bbe3221f956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:12.018430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd16dd1-1e71-42f6-a63b-4bbe3221f956 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:12.020044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 25/45] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:12.025861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56793607-d6e5-4789-a154-40a783ba2d96 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:12.131246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56793607-d6e5-4789-a154-40a783ba2d96 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:12.169162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d78f0-a918-4b20-a0bd-f2aaa18769d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:12.169823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d78f0-a918-4b20-a0bd-f2aaa18769d9 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:12.170644 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d78f0-a918-4b20-a0bd-f2aaa18769d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:12.170940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d78f0-a918-4b20-a0bd-f2aaa18769d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:12.171545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d78f0-a918-4b20-a0bd-f2aaa18769d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:12.173048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d78f0-a918-4b20-a0bd-f2aaa18769d9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:12.200034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 21/46] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:03:12 2022] POST /identity/v3/projects => generated 312 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:13.374338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abec47-c000-42fa-a1ed-43242843c261 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:13.374921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abec47-c000-42fa-a1ed-43242843c261 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:13.374921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abec47-c000-42fa-a1ed-43242843c261 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:13.376146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 26/47] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:13.381747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20efa1-6a19-4548-806c-ec147fde1c87 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:13.382198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20efa1-6a19-4548-806c-ec147fde1c87 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:13.382284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20efa1-6a19-4548-806c-ec147fde1c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:13.402748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ff20efa1-6a19-4548-806c-ec147fde1c87 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:13.404870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ff20efa1-6a19-4548-806c-ec147fde1c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:13.456212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:13.480400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 22/48] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:13.756327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abec47-c000-42fa-a1ed-43242843c261 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:13.864721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abec47-c000-42fa-a1ed-43242843c261 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:13.900965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163342b-080e-4fce-9de3-91fe0f517332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:13.901596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163342b-080e-4fce-9de3-91fe0f517332 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:13.901876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163342b-080e-4fce-9de3-91fe0f517332 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:13.902150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163342b-080e-4fce-9de3-91fe0f517332 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:13.912983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163342b-080e-4fce-9de3-91fe0f517332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-a163342b-080e-4fce-9de3-91fe0f517332 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:13.917834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:13.919207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:13.921536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 27/49] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:13.925198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20efa1-6a19-4548-806c-ec147fde1c87 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:14.031217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20efa1-6a19-4548-806c-ec147fde1c87 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:14.073282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5eb4ab-b332-4627-8b6e-29ea39789c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:14.073901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eb4ab-b332-4627-8b6e-29ea39789c61 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:14.074189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eb4ab-b332-4627-8b6e-29ea39789c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:14.074468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eb4ab-b332-4627-8b6e-29ea39789c61 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:14.075177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5eb4ab-b332-4627-8b6e-29ea39789c61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:14.075482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5eb4ab-b332-4627-8b6e-29ea39789c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:14.077439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5eb4ab-b332-4627-8b6e-29ea39789c61 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:14.086542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 23/50] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:14.093543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163342b-080e-4fce-9de3-91fe0f517332 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:14.129445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:14.209837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b88e3-4697-45c4-a078-b4b0b46f0608 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:14.210172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b88e3-4697-45c4-a078-b4b0b46f0608 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:14.210403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b88e3-4697-45c4-a078-b4b0b46f0608 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:14.211405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163342b-080e-4fce-9de3-91fe0f517332 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:14.211887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 24/51] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:03:14.216775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432730f3-73d8-4b75-bdb1-5f561dda1a3e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:14.216951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432730f3-73d8-4b75-bdb1-5f561dda1a3e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:14.217442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432730f3-73d8-4b75-bdb1-5f561dda1a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:14.237794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-432730f3-73d8-4b75-bdb1-5f561dda1a3e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:14.239018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-432730f3-73d8-4b75-bdb1-5f561dda1a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:14.255429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb9a65-00d0-45cc-b058-7438eaca2d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:14.256079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb9a65-00d0-45cc-b058-7438eaca2d9b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:14.256363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb9a65-00d0-45cc-b058-7438eaca2d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:14.256679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb9a65-00d0-45cc-b058-7438eaca2d9b None admin] PATH_INFO: `/v3/users/60fc2717779e40a0a55137c62461f753` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:14.257872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb9a65-00d0-45cc-b058-7438eaca2d9b None admin] RBAC: Authorizing `identity:get_user(user_id=60fc2717779e40a0a55137c62461f753)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:14.259364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb9a65-00d0-45cc-b058-7438eaca2d9b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:14.261191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 28/52] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:14 2022] GET /identity/v3/users/60fc2717779e40a0a55137c62461f753 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:14.268447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb9a65-00d0-45cc-b058-7438eaca2d9b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:14.284611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:14.376718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:14.381562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb9a65-00d0-45cc-b058-7438eaca2d9b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:14.422590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd070-9500-4ca9-b28d-58c3e7bf8d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:14.423237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd070-9500-4ca9-b28d-58c3e7bf8d34 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:14.423529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd070-9500-4ca9-b28d-58c3e7bf8d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:14.423911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd070-9500-4ca9-b28d-58c3e7bf8d34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:14.426617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bd070-9500-4ca9-b28d-58c3e7bf8d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:14.428341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bd070-9500-4ca9-b28d-58c3e7bf8d34 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:14.429985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 29/53] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:14.433846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 25/54] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:14.435956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd070-9500-4ca9-b28d-58c3e7bf8d34 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:14.548950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd070-9500-4ca9-b28d-58c3e7bf8d34 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:14.595443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:14.596063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:14.596346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:14.596650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:14.597833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.domain.id=default, user.id=60fc2717779e40a0a55137c62461f753) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:14.597833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:14.599153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:14.605305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 30/55] 10.222.0.95 () {56 vars in 1322 bytes} [Wed Aug 31 19:03:14 2022] GET /identity/v3/role_assignments?user.id=60fc2717779e40a0a55137c62461f753&scope.domain.id=default&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 530 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:14.738777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432730f3-73d8-4b75-bdb1-5f561dda1a3e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:14.867827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432730f3-73d8-4b75-bdb1-5f561dda1a3e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:14.903294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5dbe6b-3188-4585-a4ac-38352e2cd75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:14.903922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dbe6b-3188-4585-a4ac-38352e2cd75d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:14.904356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dbe6b-3188-4585-a4ac-38352e2cd75d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:14.904655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dbe6b-3188-4585-a4ac-38352e2cd75d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:14.905882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5dbe6b-3188-4585-a4ac-38352e2cd75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:14.907340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5dbe6b-3188-4585-a4ac-38352e2cd75d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:14.908934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 26/56] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:14.914864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:14.948157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:15.023555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770e785-2496-4c45-b9ff-edfb7064a2fe None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:15.065073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1fbba-afcc-4b0a-a1bb-97636517f753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:15.065691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1fbba-afcc-4b0a-a1bb-97636517f753 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:15.065978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1fbba-afcc-4b0a-a1bb-97636517f753 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:15.066254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1fbba-afcc-4b0a-a1bb-97636517f753 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:15.066948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1fbba-afcc-4b0a-a1bb-97636517f753 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:15.068431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1fbba-afcc-4b0a-a1bb-97636517f753 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:15.104526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 31/57] 10.222.0.95 () {60 vars in 1140 bytes} [Wed Aug 31 19:03:14 2022] POST /identity/v3/projects => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:17.074698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa431a-463f-4dc9-b2ce-e3627d7dbc95 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:17.075228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa431a-463f-4dc9-b2ce-e3627d7dbc95 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:17.075228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa431a-463f-4dc9-b2ce-e3627d7dbc95 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:17.076578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 27/58] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:17.082416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0ff3f-a28a-45b2-9e39-5b8a44e11509 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:17.082833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0ff3f-a28a-45b2-9e39-5b8a44e11509 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:17.083036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0ff3f-a28a-45b2-9e39-5b8a44e11509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:17.101372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4cb0ff3f-a28a-45b2-9e39-5b8a44e11509 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:17.102577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4cb0ff3f-a28a-45b2-9e39-5b8a44e11509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:17.169697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 32/59] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:17.450254 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa431a-463f-4dc9-b2ce-e3627d7dbc95 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:17.494496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:17.560920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa431a-463f-4dc9-b2ce-e3627d7dbc95 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:17.596560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c306e-6334-4bce-9a28-4ecccb0e31c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:17.597209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c306e-6334-4bce-9a28-4ecccb0e31c2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:17.597489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c306e-6334-4bce-9a28-4ecccb0e31c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:17.597757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c306e-6334-4bce-9a28-4ecccb0e31c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:17.598859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c306e-6334-4bce-9a28-4ecccb0e31c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:17.600292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c306e-6334-4bce-9a28-4ecccb0e31c2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:17.601888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 28/60] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:17.608138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0ff3f-a28a-45b2-9e39-5b8a44e11509 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:17.623075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:17.713561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0ff3f-a28a-45b2-9e39-5b8a44e11509 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:17.747839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c622045-cc78-4ab8-927e-147632d595ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:17.748460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622045-cc78-4ab8-927e-147632d595ad None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:17.748768 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622045-cc78-4ab8-927e-147632d595ad None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:17.749044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c622045-cc78-4ab8-927e-147632d595ad None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:17.749783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c622045-cc78-4ab8-927e-147632d595ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:17.751264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c622045-cc78-4ab8-927e-147632d595ad None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:17.777738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 33/61] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:03:17 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:21.053842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f247da1-2ff3-4bfe-908f-5a96424b2c0f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.054851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f247da1-2ff3-4bfe-908f-5a96424b2c0f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.055330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f247da1-2ff3-4bfe-908f-5a96424b2c0f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.056901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 29/62] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:21.063035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a571b-0bb5-4af7-8abf-cb40de51500a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.063505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a571b-0bb5-4af7-8abf-cb40de51500a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.063595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a571b-0bb5-4af7-8abf-cb40de51500a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.089836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-800a571b-0bb5-4af7-8abf-cb40de51500a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:21.091113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-800a571b-0bb5-4af7-8abf-cb40de51500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:21.135380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc55f44-4e2f-483d-8016-3a32bf735f79 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.136044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc55f44-4e2f-483d-8016-3a32bf735f79 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.142589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc55f44-4e2f-483d-8016-3a32bf735f79 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.144362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 30/63] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:21.150045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20fc28-792f-40cc-9d43-484ed9de9936 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.150590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20fc28-792f-40cc-9d43-484ed9de9936 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.151089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20fc28-792f-40cc-9d43-484ed9de9936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.177955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:21.182336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4b20fc28-792f-40cc-9d43-484ed9de9936 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:21.183552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4b20fc28-792f-40cc-9d43-484ed9de9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:21.188289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 34/64] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:21.322480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 31/65] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:21.475500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a571b-0bb5-4af7-8abf-cb40de51500a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:21.600717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252533-6fd9-422c-be9f-3134e1e817f0 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.601459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252533-6fd9-422c-be9f-3134e1e817f0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.603314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252533-6fd9-422c-be9f-3134e1e817f0 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.605843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 32/66] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:21.618590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c042e4-f80a-433b-b7d4-b48cb8c8d429 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.619153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c042e4-f80a-433b-b7d4-b48cb8c8d429 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.619612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c042e4-f80a-433b-b7d4-b48cb8c8d429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.642309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a571b-0bb5-4af7-8abf-cb40de51500a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:21.645112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-69c042e4-f80a-433b-b7d4-b48cb8c8d429 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:21.646566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-69c042e4-f80a-433b-b7d4-b48cb8c8d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:21.688178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47b98f-2880-4254-aa7d-5203713fe02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:21.689195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b98f-2880-4254-aa7d-5203713fe02e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.689968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b98f-2880-4254-aa7d-5203713fe02e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.690710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b98f-2880-4254-aa7d-5203713fe02e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.694985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:21.707743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47b98f-2880-4254-aa7d-5203713fe02e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-9a47b98f-2880-4254-aa7d-5203713fe02e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:21.712718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:21.713713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:21.714520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 35/67] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:03:21.715879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3b9e-89e7-458e-9567-83eb79e7d026 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.715879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3b9e-89e7-458e-9567-83eb79e7d026 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.716171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3b9e-89e7-458e-9567-83eb79e7d026 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.717839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 36/68] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:21.718495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf3b9e-89e7-458e-9567-83eb79e7d026 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:21.751794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 33/69] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 31 19:03:21.760731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c042e4-f80a-433b-b7d4-b48cb8c8d429 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:21.791189 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:21.841480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf3b9e-89e7-458e-9567-83eb79e7d026 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:21.890548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436efa6d-90a3-4546-81c3-b299cbb2a7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:21.891427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436efa6d-90a3-4546-81c3-b299cbb2a7ef None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.891979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436efa6d-90a3-4546-81c3-b299cbb2a7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.892536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436efa6d-90a3-4546-81c3-b299cbb2a7ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.896932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c042e4-f80a-433b-b7d4-b48cb8c8d429 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:21.904947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436efa6d-90a3-4546-81c3-b299cbb2a7ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-436efa6d-90a3-4546-81c3-b299cbb2a7ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:21.909760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:21.910723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 37/70] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:21.912122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0536c13-4b84-4a95-ab2d-730bd622083a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.913527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0536c13-4b84-4a95-ab2d-730bd622083a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.913527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0536c13-4b84-4a95-ab2d-730bd622083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.936992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f0536c13-4b84-4a95-ab2d-730bd622083a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:21.938290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f0536c13-4b84-4a95-ab2d-730bd622083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:21.948336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1978f-669f-474e-a344-7f680c7880b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:21.949000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1978f-669f-474e-a344-7f680c7880b9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.949286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1978f-669f-474e-a344-7f680c7880b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.949559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1978f-669f-474e-a344-7f680c7880b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.950236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1978f-669f-474e-a344-7f680c7880b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:21.950545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1978f-669f-474e-a344-7f680c7880b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:21.953095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1978f-669f-474e-a344-7f680c7880b9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:21.963487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 34/71] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:21.964760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df66a7a-f39c-43be-b6ec-259599d0adb3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:21.965062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df66a7a-f39c-43be-b6ec-259599d0adb3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:21.965345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df66a7a-f39c-43be-b6ec-259599d0adb3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:21.972468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 35/72] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:21.972468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df66a7a-f39c-43be-b6ec-259599d0adb3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.002529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:22.030142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 38/73] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:22.031485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0536c13-4b84-4a95-ab2d-730bd622083a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.095326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:22.106007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df66a7a-f39c-43be-b6ec-259599d0adb3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.146887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c3049-6ba3-4dbb-baaf-1ee68ea298de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.147850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3049-6ba3-4dbb-baaf-1ee68ea298de None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.147850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3049-6ba3-4dbb-baaf-1ee68ea298de None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.148136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3049-6ba3-4dbb-baaf-1ee68ea298de None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.148770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c3049-6ba3-4dbb-baaf-1ee68ea298de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:22.149072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c3049-6ba3-4dbb-baaf-1ee68ea298de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:22.150796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c3049-6ba3-4dbb-baaf-1ee68ea298de None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:22.160826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 36/74] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:22.161859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea90630-6499-45a1-a0b4-88c9aaafb7c1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.162206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea90630-6499-45a1-a0b4-88c9aaafb7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.162487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea90630-6499-45a1-a0b4-88c9aaafb7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.166307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0536c13-4b84-4a95-ab2d-730bd622083a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.188439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-aea90630-6499-45a1-a0b4-88c9aaafb7c1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:22.189615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aea90630-6499-45a1-a0b4-88c9aaafb7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:22.213434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2d57b-78cf-4310-a907-9395188ece43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.214699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2d57b-78cf-4310-a907-9395188ece43 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.215331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2d57b-78cf-4310-a907-9395188ece43 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.215754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2d57b-78cf-4310-a907-9395188ece43 None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.235230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2d57b-78cf-4310-a907-9395188ece43 None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:22.237366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2d57b-78cf-4310-a907-9395188ece43 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:22.239672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 39/75] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:22 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:22.246234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2d57b-78cf-4310-a907-9395188ece43 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.278042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 37/76] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:22.279708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea90630-6499-45a1-a0b4-88c9aaafb7c1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.318426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:22.399823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2d57b-78cf-4310-a907-9395188ece43 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.411191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:22.436843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea90630-6499-45a1-a0b4-88c9aaafb7c1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.442405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1bafa-b112-48f6-83ee-a84b707ca93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.443049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1bafa-b112-48f6-83ee-a84b707ca93e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.443325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1bafa-b112-48f6-83ee-a84b707ca93e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.443595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1bafa-b112-48f6-83ee-a84b707ca93e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.455603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1bafa-b112-48f6-83ee-a84b707ca93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-dcd1bafa-b112-48f6-83ee-a84b707ca93e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:22.460544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:22.461414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 40/77] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:03:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:22.462627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1bafa-b112-48f6-83ee-a84b707ca93e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.487506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabe0a1-0129-4ad1-b70c-5ff503e17d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.488346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabe0a1-0129-4ad1-b70c-5ff503e17d2d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.488666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabe0a1-0129-4ad1-b70c-5ff503e17d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.488938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabe0a1-0129-4ad1-b70c-5ff503e17d2d None admin] PATH_INFO: `/v3/users/60fc2717779e40a0a55137c62461f753` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.490070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabe0a1-0129-4ad1-b70c-5ff503e17d2d None admin] RBAC: Authorizing `identity:get_user(user_id=60fc2717779e40a0a55137c62461f753)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:22.491680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabe0a1-0129-4ad1-b70c-5ff503e17d2d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:22.500875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 38/78] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:22 2022] GET /identity/v3/users/60fc2717779e40a0a55137c62461f753 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:22.500875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabe0a1-0129-4ad1-b70c-5ff503e17d2d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.596772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:22.607777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabe0a1-0129-4ad1-b70c-5ff503e17d2d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.632901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1bafa-b112-48f6-83ee-a84b707ca93e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.646875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27d902-75d5-47e2-8cbb-f2c1a6aaccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.647507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d902-75d5-47e2-8cbb-f2c1a6aaccd8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.647785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d902-75d5-47e2-8cbb-f2c1a6aaccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.648058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d902-75d5-47e2-8cbb-f2c1a6aaccd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.658399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27d902-75d5-47e2-8cbb-f2c1a6aaccd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-af27d902-75d5-47e2-8cbb-f2c1a6aaccd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:22.663372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:22.664271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:22.665054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 39/79] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:22.666243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27d902-75d5-47e2-8cbb-f2c1a6aaccd8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.673732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e50e2-c356-48c1-aeb4-a4db24d5da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.674353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e50e2-c356-48c1-aeb4-a4db24d5da31 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.674638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e50e2-c356-48c1-aeb4-a4db24d5da31 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.674912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e50e2-c356-48c1-aeb4-a4db24d5da31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.709669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:22.743727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e50e2-c356-48c1-aeb4-a4db24d5da31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:22.745257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e50e2-c356-48c1-aeb4-a4db24d5da31 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:22.781349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27d902-75d5-47e2-8cbb-f2c1a6aaccd8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.819307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.820063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.820451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.820883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.821646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:22.822049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:22.823540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:22.831470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 40/80] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:03:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:22.832261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.847458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:22.859114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 41/81] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:22.860129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e50e2-c356-48c1-aeb4-a4db24d5da31 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:22.927346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:22.934814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac3aa0-3417-4f3a-acb4-7083e1171b33 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.961578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e50e2-c356-48c1-aeb4-a4db24d5da31 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:22.970793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f3244-1476-42ae-b3fb-ba771a7c4a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.971405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f3244-1476-42ae-b3fb-ba771a7c4a3c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.971697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f3244-1476-42ae-b3fb-ba771a7c4a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.971975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f3244-1476-42ae-b3fb-ba771a7c4a3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.996362 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a46ee9-35a1-404a-9722-fd04a590a6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:22.997108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a46ee9-35a1-404a-9722-fd04a590a6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:22.997394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a46ee9-35a1-404a-9722-fd04a590a6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:22.997697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a46ee9-35a1-404a-9722-fd04a590a6e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:22.998811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a46ee9-35a1-404a-9722-fd04a590a6e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.000284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a46ee9-35a1-404a-9722-fd04a590a6e7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.004668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 42/82] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:23.004668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a46ee9-35a1-404a-9722-fd04a590a6e7 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:23.043768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f3244-1476-42ae-b3fb-ba771a7c4a3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.045947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f3244-1476-42ae-b3fb-ba771a7c4a3c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.060617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:23.128350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a46ee9-35a1-404a-9722-fd04a590a6e7 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:23.155356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 41/83] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:23.156260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f3244-1476-42ae-b3fb-ba771a7c4a3c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:23.169611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:23.171446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:23.171731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:23.172009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:23.172731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] RBAC: Adding query filter params (name=member) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:23.173039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.174469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.177671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:23.188641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 43/84] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:23.188641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:23.288639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f3244-1476-42ae-b3fb-ba771a7c4a3c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:23.303793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:23.306393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39ed1-6410-43af-8f9e-913452cd8192 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:23.330830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682e28b-8d8d-444f-9fd5-696f23588f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:23.331610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682e28b-8d8d-444f-9fd5-696f23588f80 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:23.332064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682e28b-8d8d-444f-9fd5-696f23588f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:23.332443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682e28b-8d8d-444f-9fd5-696f23588f80 None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:23.333835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682e28b-8d8d-444f-9fd5-696f23588f80 None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.335613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682e28b-8d8d-444f-9fd5-696f23588f80 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.337526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 42/85] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:23 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:23.338499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682e28b-8d8d-444f-9fd5-696f23588f80 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:23.348059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc7109-9800-45d0-ad81-c72bbdb3884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:23.348739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc7109-9800-45d0-ad81-c72bbdb3884d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:23.349025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc7109-9800-45d0-ad81-c72bbdb3884d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:23.349304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc7109-9800-45d0-ad81-c72bbdb3884d None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:23.350460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc7109-9800-45d0-ad81-c72bbdb3884d None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.351944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc7109-9800-45d0-ad81-c72bbdb3884d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.353653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 44/86] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:23 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:23.354443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc7109-9800-45d0-ad81-c72bbdb3884d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:23.444047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:23.459461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682e28b-8d8d-444f-9fd5-696f23588f80 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:23.488165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc7109-9800-45d0-ad81-c72bbdb3884d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:23.495970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465589f9-712c-4783-a368-d6d5c50a43d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:23.496607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465589f9-712c-4783-a368-d6d5c50a43d1 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:23.496894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465589f9-712c-4783-a368-d6d5c50a43d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:23.497173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465589f9-712c-4783-a368-d6d5c50a43d1 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:23.498031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465589f9-712c-4783-a368-d6d5c50a43d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.498031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465589f9-712c-4783-a368-d6d5c50a43d1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.526524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f865-c95e-4551-af76-9a635c7fa19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:23.527150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f865-c95e-4551-af76-9a635c7fa19c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:23.527430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f865-c95e-4551-af76-9a635c7fa19c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:23.527702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f865-c95e-4551-af76-9a635c7fa19c None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:23.527831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 43/87] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:03:23 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:23.528956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465589f9-712c-4783-a368-d6d5c50a43d1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:23.529713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713f865-c95e-4551-af76-9a635c7fa19c None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.531681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713f865-c95e-4551-af76-9a635c7fa19c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.533477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 45/88] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:23 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:23.534631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f865-c95e-4551-af76-9a635c7fa19c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:23.581266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:23.672528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465589f9-712c-4783-a368-d6d5c50a43d1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:23.690513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f865-c95e-4551-af76-9a635c7fa19c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:23.724159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaad916-c36d-4b53-a973-2e5eb45267fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:23.724849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaad916-c36d-4b53-a973-2e5eb45267fe None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:23.725225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaad916-c36d-4b53-a973-2e5eb45267fe None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:23.725468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaad916-c36d-4b53-a973-2e5eb45267fe None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:23.727247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaad916-c36d-4b53-a973-2e5eb45267fe None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.728066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaad916-c36d-4b53-a973-2e5eb45267fe None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.729771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 44/89] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:23 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:23.730554 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaad916-c36d-4b53-a973-2e5eb45267fe None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:23.742123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23493e-492f-4a40-8820-232a751e17da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:23.742123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23493e-492f-4a40-8820-232a751e17da None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:23.742123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23493e-492f-4a40-8820-232a751e17da None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:23.742602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23493e-492f-4a40-8820-232a751e17da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:23.748798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:23.856462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23493e-492f-4a40-8820-232a751e17da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.859462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23493e-492f-4a40-8820-232a751e17da None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.893549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaad916-c36d-4b53-a973-2e5eb45267fe None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:23.896671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:23.945961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:23.946971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:23.947477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:23.947991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:23.948971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] RBAC: Adding query filter params (role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231, user.id=99d2701b53de47caa5ad2c67fb1f0e64) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:23.949483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:23.951916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:23.960557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 45/90] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:23 2022] GET /identity/v3/role_assignments?user.id=99d2701b53de47caa5ad2c67fb1f0e64&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:23.961815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:24.037403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 46/91] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:24.038843 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23493e-492f-4a40-8820-232a751e17da None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:24.083803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78700a57-9193-40c5-8f18-6b90ef0e4a03 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:24.127722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:24.128641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:24.129132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:24.129685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:24.130676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231, user.id=60fc2717779e40a0a55137c62461f753) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:24.131226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:24.133075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:24.140933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 46/92] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:23 2022] GET /identity/v3/role_assignments?user.id=60fc2717779e40a0a55137c62461f753&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:24.144562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:24.176074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:24.204599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:24.339603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343030d-2e24-4d86-8905-7c0d0dddf534 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:24.390717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bd4f3-2bf4-4f40-9739-679961537d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:24.391430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd4f3-2bf4-4f40-9739-679961537d16 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:24.391717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd4f3-2bf4-4f40-9739-679961537d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:24.391994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bd4f3-2bf4-4f40-9739-679961537d16 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:24.393227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bd4f3-2bf4-4f40-9739-679961537d16 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:24.394867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bd4f3-2bf4-4f40-9739-679961537d16 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:24.397162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 47/93] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:24 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:24.399015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23493e-492f-4a40-8820-232a751e17da None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:24.404290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bd4f3-2bf4-4f40-9739-679961537d16 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:24.488179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbc33d-028a-4629-a855-09559c680d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:24.489234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc33d-028a-4629-a855-09559c680d5e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:24.489636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc33d-028a-4629-a855-09559c680d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:24.489951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc33d-028a-4629-a855-09559c680d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:24.642034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:24.646020 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bd4f3-2bf4-4f40-9739-679961537d16 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:24.652488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbc33d-028a-4629-a855-09559c680d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:24.654769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbc33d-028a-4629-a855-09559c680d5e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:24.730596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:24.732447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:24.733087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:24.733355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:24.733660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:24.734422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] RBAC: Adding query filter params (role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231, user.id=99d2701b53de47caa5ad2c67fb1f0e64) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:24.734833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:24.736214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:24.751355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 48/94] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:24 2022] GET /identity/v3/role_assignments?user.id=99d2701b53de47caa5ad2c67fb1f0e64&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=79dbf286cf884619af62f33ea75c52bf => generated 259 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:24.851103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 47/95] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 813 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:24.856508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:24.975496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b10a2-80e1-476c-b937-cff1eb1441d7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:25.012815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ee8c1-0b3b-45ca-b7e0-2820ad12a575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:25.013645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee8c1-0b3b-45ca-b7e0-2820ad12a575 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:25.014275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee8c1-0b3b-45ca-b7e0-2820ad12a575 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:25.014275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee8c1-0b3b-45ca-b7e0-2820ad12a575 None admin] PATH_INFO: `/v3/projects/1bd0f7ac351d4a14b83c0934aa4ca75a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:25.015421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ee8c1-0b3b-45ca-b7e0-2820ad12a575 None admin] RBAC: Authorizing `identity:get_project(project_id=1bd0f7ac351d4a14b83c0934aa4ca75a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:25.016941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ee8c1-0b3b-45ca-b7e0-2820ad12a575 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:25.019012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 49/96] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:24 2022] GET /identity/v3/projects/1bd0f7ac351d4a14b83c0934aa4ca75a => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:25.025005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbc33d-028a-4629-a855-09559c680d5e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:25.167643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbc33d-028a-4629-a855-09559c680d5e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:25.211364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a71e44-9854-4958-8e4a-f050c68cef14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:25.212791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71e44-9854-4958-8e4a-f050c68cef14 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:25.213275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71e44-9854-4958-8e4a-f050c68cef14 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:25.213741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71e44-9854-4958-8e4a-f050c68cef14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:25.214785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a71e44-9854-4958-8e4a-f050c68cef14 None admin] RBAC: Adding query filter params (role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=1bd0f7ac351d4a14b83c0934aa4ca75a, user.id=99d2701b53de47caa5ad2c67fb1f0e64) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:25.215289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a71e44-9854-4958-8e4a-f050c68cef14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:25.217366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a71e44-9854-4958-8e4a-f050c68cef14 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:25.220001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:25.224784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 48/97] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:25 2022] GET /identity/v3/role_assignments?user.id=99d2701b53de47caa5ad2c67fb1f0e64&scope.project.id=1bd0f7ac351d4a14b83c0934aa4ca75a&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:26.620122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d67021-5a81-434c-9888-aad45ddea33b None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:26.621125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d67021-5a81-434c-9888-aad45ddea33b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:26.621599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d67021-5a81-434c-9888-aad45ddea33b None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:26.623652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 50/98] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:26.629577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be076be2-ac9a-4344-9b9a-6dd0273235eb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:26.630450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be076be2-ac9a-4344-9b9a-6dd0273235eb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:26.630924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be076be2-ac9a-4344-9b9a-6dd0273235eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:26.650467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-be076be2-ac9a-4344-9b9a-6dd0273235eb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:26.652000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-be076be2-ac9a-4344-9b9a-6dd0273235eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:26.710263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aac9f-8462-4fae-9a44-9886b5bdc670 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:26.711004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aac9f-8462-4fae-9a44-9886b5bdc670 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:26.711609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aac9f-8462-4fae-9a44-9886b5bdc670 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:26.713927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 51/99] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:03:26.722086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390f0a8-0bce-4fd6-9a21-ef01b52be7a2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:26.722923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390f0a8-0bce-4fd6-9a21-ef01b52be7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:26.723704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390f0a8-0bce-4fd6-9a21-ef01b52be7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:26.745626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3390f0a8-0bce-4fd6-9a21-ef01b52be7a2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:26.746985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 49/100] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:26.747725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3390f0a8-0bce-4fd6-9a21-ef01b52be7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:26.800920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:26.835973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 52/101] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:26.962572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d17f0c-20aa-40ae-884a-6755d21254e5 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:26.962772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d17f0c-20aa-40ae-884a-6755d21254e5 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:26.963048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d17f0c-20aa-40ae-884a-6755d21254e5 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:26.964397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 50/102] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:26.970312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308e0ed-6e1e-4b18-bf8d-f42fe13a4e2b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:26.970606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308e0ed-6e1e-4b18-bf8d-f42fe13a4e2b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:26.970886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308e0ed-6e1e-4b18-bf8d-f42fe13a4e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:26.993794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6308e0ed-6e1e-4b18-bf8d-f42fe13a4e2b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:26.995000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6308e0ed-6e1e-4b18-bf8d-f42fe13a4e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:27.069733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 53/103] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:27.090030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bfb13-ed9c-413d-aae7-7fe0f5fbdba0 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:27.090309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bfb13-ed9c-413d-aae7-7fe0f5fbdba0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:27.090626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bfb13-ed9c-413d-aae7-7fe0f5fbdba0 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:27.092172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 51/104] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:27.097503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59951c6-ea55-4e7f-95e5-42c825872134 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:27.097797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59951c6-ea55-4e7f-95e5-42c825872134 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:27.098079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59951c6-ea55-4e7f-95e5-42c825872134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:27.117341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b59951c6-ea55-4e7f-95e5-42c825872134 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:27.118547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b59951c6-ea55-4e7f-95e5-42c825872134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:27.201739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 54/105] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:27.214827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bfb13-ed9c-413d-aae7-7fe0f5fbdba0 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:27.265746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59951c6-ea55-4e7f-95e5-42c825872134 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:27.301858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:27.344599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bfb13-ed9c-413d-aae7-7fe0f5fbdba0 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:27.382441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:27.390570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fefa4-cc23-451f-90b6-524444e29c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:27.390743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59951c6-ea55-4e7f-95e5-42c825872134 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:27.391278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fefa4-cc23-451f-90b6-524444e29c57 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:27.391562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fefa4-cc23-451f-90b6-524444e29c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:27.391933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fefa4-cc23-451f-90b6-524444e29c57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:27.411942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fefa4-cc23-451f-90b6-524444e29c57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-1c1fefa4-cc23-451f-90b6-524444e29c57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:27.421700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:27.422578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:27.423327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:27.427496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 52/106] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:27.442065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb7427-7fd1-4c68-9ff8-063d450c1378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:27.442783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb7427-7fd1-4c68-9ff8-063d450c1378 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:27.443080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb7427-7fd1-4c68-9ff8-063d450c1378 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:27.443364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb7427-7fd1-4c68-9ff8-063d450c1378 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:27.446340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fefa4-cc23-451f-90b6-524444e29c57 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:27.472407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb7427-7fd1-4c68-9ff8-063d450c1378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c7bb7427-7fd1-4c68-9ff8-063d450c1378 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:27.479455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:27.480292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:27.482649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:27.488793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 55/107] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:03:27.489661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb7427-7fd1-4c68-9ff8-063d450c1378 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:27.614348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fefa4-cc23-451f-90b6-524444e29c57 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:27.659543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:27.660173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:27.660439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:27.660905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:27.662246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:27.663127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:27.665340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:27.666663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:27.682835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 53/108] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:27.683820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:27.690379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb7427-7fd1-4c68-9ff8-063d450c1378 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:27.728287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:27.729162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:27.729592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:27.730034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:27.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:27.731505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:27.735110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:27.741325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:27.745137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 56/109] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:27.746894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:27.801447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6c0a8-072f-42e9-bc37-acedb79b5561 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:27.839366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56448e0-0dda-4801-a63b-67efd1424000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:27.840002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56448e0-0dda-4801-a63b-67efd1424000 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:27.840481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56448e0-0dda-4801-a63b-67efd1424000 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:27.840935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56448e0-0dda-4801-a63b-67efd1424000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:27.855246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56448e0-0dda-4801-a63b-67efd1424000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-b56448e0-0dda-4801-a63b-67efd1424000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:27.861890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:27.862782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:27.863560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 54/110] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:27.864224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56448e0-0dda-4801-a63b-67efd1424000 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:27.873763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1981d7-510a-4d3b-942c-c4a325b00031 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:27.913217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742441e-6ffc-45ce-a3b2-aece2bc1b016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:27.913890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742441e-6ffc-45ce-a3b2-aece2bc1b016 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:27.914175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742441e-6ffc-45ce-a3b2-aece2bc1b016 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:27.914447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742441e-6ffc-45ce-a3b2-aece2bc1b016 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:27.915593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742441e-6ffc-45ce-a3b2-aece2bc1b016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:27.917370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742441e-6ffc-45ce-a3b2-aece2bc1b016 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:27.919253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 57/111] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:27.919888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742441e-6ffc-45ce-a3b2-aece2bc1b016 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:27.941648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:27.992509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56448e0-0dda-4801-a63b-67efd1424000 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:28.004185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:28.032428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43221f-a965-42bf-9404-596d03d96110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:28.033109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43221f-a965-42bf-9404-596d03d96110 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.033382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43221f-a965-42bf-9404-596d03d96110 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.033682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43221f-a965-42bf-9404-596d03d96110 None admin] PATH_INFO: `/v3/users/d2ba1cc0a455447ab883edd9eb275586` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.048129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43221f-a965-42bf-9404-596d03d96110 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ba1cc0a455447ab883edd9eb275586)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:28.049781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43221f-a965-42bf-9404-596d03d96110 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:28.051719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 55/112] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:27 2022] GET /identity/v3/users/d2ba1cc0a455447ab883edd9eb275586 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:28.053039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ffa24-f9d4-411b-86c7-e08159e7221c None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.053319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ffa24-f9d4-411b-86c7-e08159e7221c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.053587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ffa24-f9d4-411b-86c7-e08159e7221c None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.055004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 56/113] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:28.059184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ffa24-f9d4-411b-86c7-e08159e7221c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:28.094303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742441e-6ffc-45ce-a3b2-aece2bc1b016 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:28.141992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d234533-97d2-468b-adc6-11aeffd3bc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:28.142642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d234533-97d2-468b-adc6-11aeffd3bc2e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.142924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d234533-97d2-468b-adc6-11aeffd3bc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.143682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d234533-97d2-468b-adc6-11aeffd3bc2e None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.145316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d234533-97d2-468b-adc6-11aeffd3bc2e None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:28.147103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d234533-97d2-468b-adc6-11aeffd3bc2e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:28.154751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 58/114] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:27 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:28.154751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d234533-97d2-468b-adc6-11aeffd3bc2e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:28.171698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ffa24-f9d4-411b-86c7-e08159e7221c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:28.210620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:28.211230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.211525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.211786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.212459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:28.212785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:28.214213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:28.229110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 57/115] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:28.229110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:28.260043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:28.268717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:28.292982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d234533-97d2-468b-adc6-11aeffd3bc2e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:28.335476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695451a-4bab-4a75-a5f8-182fd498f071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:28.336089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695451a-4bab-4a75-a5f8-182fd498f071 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.336377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695451a-4bab-4a75-a5f8-182fd498f071 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.336688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695451a-4bab-4a75-a5f8-182fd498f071 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.337347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695451a-4bab-4a75-a5f8-182fd498f071 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:28.338808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695451a-4bab-4a75-a5f8-182fd498f071 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:28.376615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 59/116] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:03:28 2022] POST /identity/v3/projects => generated 302 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:28.376615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15e6db-2fd1-4289-ae97-95f7f8e4b4b0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.376615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15e6db-2fd1-4289-ae97-95f7f8e4b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.376615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15e6db-2fd1-4289-ae97-95f7f8e4b4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.396718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2e15e6db-2fd1-4289-ae97-95f7f8e4b4b0 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:28.397530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770693fe-5777-44b1-a891-6a104f4ee555 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:28.398914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2e15e6db-2fd1-4289-ae97-95f7f8e4b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:28.442081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0b7ab-1873-46cd-b2fa-d823b3a4c613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:28.443005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0b7ab-1873-46cd-b2fa-d823b3a4c613 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.443294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0b7ab-1873-46cd-b2fa-d823b3a4c613 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.443574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0b7ab-1873-46cd-b2fa-d823b3a4c613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.534544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0b7ab-1873-46cd-b2fa-d823b3a4c613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:28.535641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 60/117] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:28.536014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0b7ab-1873-46cd-b2fa-d823b3a4c613 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:28.537404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15e6db-2fd1-4289-ae97-95f7f8e4b4b0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:28.581848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:28.627539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:28.671333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 58/118] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:28.673020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0b7ab-1873-46cd-b2fa-d823b3a4c613 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:28.691152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15e6db-2fd1-4289-ae97-95f7f8e4b4b0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:28.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef32239-77cc-4040-b024-6b95b317beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:28.744938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef32239-77cc-4040-b024-6b95b317beed None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.745683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef32239-77cc-4040-b024-6b95b317beed None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.746183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef32239-77cc-4040-b024-6b95b317beed None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.748478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef32239-77cc-4040-b024-6b95b317beed None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:28.750423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef32239-77cc-4040-b024-6b95b317beed None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:28.752899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 61/119] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:28 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:28.754468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed32cc-3e16-4f6b-bb1e-119d6dfc8cb2 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.754984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed32cc-3e16-4f6b-bb1e-119d6dfc8cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.755455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed32cc-3e16-4f6b-bb1e-119d6dfc8cb2 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.756889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 62/120] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:28.757885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed32cc-3e16-4f6b-bb1e-119d6dfc8cb2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:28.822619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0b7ab-1873-46cd-b2fa-d823b3a4c613 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:28.855677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:28.863512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa291ec-06ae-48d0-909c-b80310f95fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:28.864258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa291ec-06ae-48d0-909c-b80310f95fbe None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.864543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa291ec-06ae-48d0-909c-b80310f95fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.864850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa291ec-06ae-48d0-909c-b80310f95fbe None admin] PATH_INFO: `/v3/users/60fc2717779e40a0a55137c62461f753` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.866141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa291ec-06ae-48d0-909c-b80310f95fbe None admin] RBAC: Authorizing `identity:get_user(user_id=60fc2717779e40a0a55137c62461f753)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:28.867675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa291ec-06ae-48d0-909c-b80310f95fbe None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:28.870649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed32cc-3e16-4f6b-bb1e-119d6dfc8cb2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:28.874975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 59/121] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:28 2022] GET /identity/v3/users/60fc2717779e40a0a55137c62461f753 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:28.875601 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa291ec-06ae-48d0-909c-b80310f95fbe None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:28.903546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:28.911145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36916db5-41a3-4985-b54f-c626db82f079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:28.911778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36916db5-41a3-4985-b54f-c626db82f079 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.912064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36916db5-41a3-4985-b54f-c626db82f079 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.912338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36916db5-41a3-4985-b54f-c626db82f079 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.913498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36916db5-41a3-4985-b54f-c626db82f079 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:28.914952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36916db5-41a3-4985-b54f-c626db82f079 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:28.916603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 63/122] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:28 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:28.921653 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa15ee-8258-4ff5-b375-f80bb9e191b7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:28.922019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa15ee-8258-4ff5-b375-f80bb9e191b7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:28.922367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa15ee-8258-4ff5-b375-f80bb9e191b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:28.943733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6dfa15ee-8258-4ff5-b375-f80bb9e191b7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:28.945061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6dfa15ee-8258-4ff5-b375-f80bb9e191b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:28.989093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa291ec-06ae-48d0-909c-b80310f95fbe None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.024998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 64/123] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:29.026077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbd24e-cbd6-4121-8cb2-4f5a0b785405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:29.026556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa15ee-8258-4ff5-b375-f80bb9e191b7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.026739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbd24e-cbd6-4121-8cb2-4f5a0b785405 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:29.027072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbd24e-cbd6-4121-8cb2-4f5a0b785405 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:29.027345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbd24e-cbd6-4121-8cb2-4f5a0b785405 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/users/99d2701b53de47caa5ad2c67fb1f0e64/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:29.030240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbd24e-cbd6-4121-8cb2-4f5a0b785405 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f379c8467d44bcbbf9cbc6aac3a3231, user_id=99d2701b53de47caa5ad2c67fb1f0e64, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:29.032550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbd24e-cbd6-4121-8cb2-4f5a0b785405 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:29.051524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 60/124] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:28 2022] PUT /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/users/99d2701b53de47caa5ad2c67fb1f0e64/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:29.051524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbd24e-cbd6-4121-8cb2-4f5a0b785405 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.125262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:29.171667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbd24e-cbd6-4121-8cb2-4f5a0b785405 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.187048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa15ee-8258-4ff5-b375-f80bb9e191b7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.214629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534b2b-2973-47c4-8738-99a3d804eb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:29.215438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534b2b-2973-47c4-8738-99a3d804eb54 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:29.215954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534b2b-2973-47c4-8738-99a3d804eb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:29.216752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534b2b-2973-47c4-8738-99a3d804eb54 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:29.218547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39534b2b-2973-47c4-8738-99a3d804eb54 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:29.220328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39534b2b-2973-47c4-8738-99a3d804eb54 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:29.222550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 61/125] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:29 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:29.223157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534b2b-2973-47c4-8738-99a3d804eb54 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.226567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:29.244932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdad634-702a-4c3c-bf8b-a356a8499eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:29.246000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdad634-702a-4c3c-bf8b-a356a8499eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:29.246559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdad634-702a-4c3c-bf8b-a356a8499eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:29.246998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdad634-702a-4c3c-bf8b-a356a8499eb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:29.262879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdad634-702a-4c3c-bf8b-a356a8499eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3bdad634-702a-4c3c-bf8b-a356a8499eb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:29.268288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:29.269214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:29.269968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 65/126] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:03:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:29.270593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdad634-702a-4c3c-bf8b-a356a8499eb3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.354114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534b2b-2973-47c4-8738-99a3d804eb54 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.384877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:29.396361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:29.397132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:29.397512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:29.397901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:29.398725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] RBAC: Adding query filter params (role.id=424ecb0deee84a068044178a8ee9ad52, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231, user.id=d2ba1cc0a455447ab883edd9eb275586) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:29.399165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:29.400787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:29.412608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 62/127] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:29 2022] GET /identity/v3/role_assignments?user.id=d2ba1cc0a455447ab883edd9eb275586&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=424ecb0deee84a068044178a8ee9ad52 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:29.412608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.421349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdad634-702a-4c3c-bf8b-a356a8499eb3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.511797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202afc90-032c-45ab-a1ca-3e7c1abf9847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:29.512517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202afc90-032c-45ab-a1ca-3e7c1abf9847 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:29.512893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202afc90-032c-45ab-a1ca-3e7c1abf9847 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:29.513716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202afc90-032c-45ab-a1ca-3e7c1abf9847 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/users/60fc2717779e40a0a55137c62461f753/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:29.516380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202afc90-032c-45ab-a1ca-3e7c1abf9847 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f379c8467d44bcbbf9cbc6aac3a3231, user_id=60fc2717779e40a0a55137c62461f753, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:29.518025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202afc90-032c-45ab-a1ca-3e7c1abf9847 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:29.539863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 66/128] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:29 2022] PUT /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/users/60fc2717779e40a0a55137c62461f753/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:29.540831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202afc90-032c-45ab-a1ca-3e7c1abf9847 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.602056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786cc54-a255-4a0b-a0db-78caf1956fc1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.640667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:29.646797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:29.647735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:29.648219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:29.648720 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:29.649648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:29.650164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:29.651911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:29.664178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 63/129] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:03:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:29.670543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.723243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202afc90-032c-45ab-a1ca-3e7c1abf9847 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.761333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:29.762054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaded2-9971-49d3-9d93-de0463a8c4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:29.762416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaded2-9971-49d3-9d93-de0463a8c4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:29.762703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaded2-9971-49d3-9d93-de0463a8c4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:29.762973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaded2-9971-49d3-9d93-de0463a8c4a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:29.779402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbaded2-9971-49d3-9d93-de0463a8c4a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1dbaded2-9971-49d3-9d93-de0463a8c4a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:29.784918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:29.785851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:29.786883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 67/130] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:29.793300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaded2-9971-49d3-9d93-de0463a8c4a3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.830453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844a7db-344e-4567-b343-ad38b62f6e3a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.878431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b02d2-40c2-4ac6-b83a-b6862f1f2c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:29.879341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b02d2-40c2-4ac6-b83a-b6862f1f2c90 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:29.879770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b02d2-40c2-4ac6-b83a-b6862f1f2c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:29.880187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b02d2-40c2-4ac6-b83a-b6862f1f2c90 None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:29.881788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b02d2-40c2-4ac6-b83a-b6862f1f2c90 None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:29.883471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b02d2-40c2-4ac6-b83a-b6862f1f2c90 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:29.885833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 64/131] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:29 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:29.896766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b02d2-40c2-4ac6-b83a-b6862f1f2c90 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:29.940297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaded2-9971-49d3-9d93-de0463a8c4a3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:29.962508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:30.014396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:30.015469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:30.016016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:30.016447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:30.017702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:30.018243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:30.020865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:30.036241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 68/132] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:30.042314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:30.051706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b02d2-40c2-4ac6-b83a-b6862f1f2c90 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:30.076118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:30.098881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2a235-bd63-4e15-97eb-fb5c6e58b707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:30.101035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a235-bd63-4e15-97eb-fb5c6e58b707 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:30.101509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a235-bd63-4e15-97eb-fb5c6e58b707 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:30.102905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a235-bd63-4e15-97eb-fb5c6e58b707 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:30.105406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2a235-bd63-4e15-97eb-fb5c6e58b707 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:30.109179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2a235-bd63-4e15-97eb-fb5c6e58b707 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:30.112445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 65/133] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:29 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:30.129438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2a235-bd63-4e15-97eb-fb5c6e58b707 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:30.282528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69831f-6bba-488f-a86a-b055cd044f3a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:30.321992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2a235-bd63-4e15-97eb-fb5c6e58b707 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:30.366855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0003a97-760f-41e5-92cb-a829da0474e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:30.367599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003a97-760f-41e5-92cb-a829da0474e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:30.367879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003a97-760f-41e5-92cb-a829da0474e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:30.368149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003a97-760f-41e5-92cb-a829da0474e6 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/users/99d2701b53de47caa5ad2c67fb1f0e64/roles/79dbf286cf884619af62f33ea75c52bf` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:30.372918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0003a97-760f-41e5-92cb-a829da0474e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f379c8467d44bcbbf9cbc6aac3a3231, user_id=99d2701b53de47caa5ad2c67fb1f0e64, role_id=79dbf286cf884619af62f33ea75c52bf)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:30.374860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0003a97-760f-41e5-92cb-a829da0474e6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:30.378434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102d985-d515-4c6c-881a-02c45906f682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:30.379072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102d985-d515-4c6c-881a-02c45906f682 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:30.379346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102d985-d515-4c6c-881a-02c45906f682 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:30.379619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102d985-d515-4c6c-881a-02c45906f682 None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:30.380804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102d985-d515-4c6c-881a-02c45906f682 None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:30.384440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102d985-d515-4c6c-881a-02c45906f682 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:30.389152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 69/134] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:30 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:30.396941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102d985-d515-4c6c-881a-02c45906f682 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:30.403066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 66/135] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:30 2022] PUT /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/users/99d2701b53de47caa5ad2c67fb1f0e64/roles/79dbf286cf884619af62f33ea75c52bf => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:03:30.478935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:30.564524 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102d985-d515-4c6c-881a-02c45906f682 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:30.616194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8259e-c64c-4239-bb73-799194207b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:30.616928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8259e-c64c-4239-bb73-799194207b1b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:30.617213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8259e-c64c-4239-bb73-799194207b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:30.617478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8259e-c64c-4239-bb73-799194207b1b None admin] PATH_INFO: `/v3/projects/1bd0f7ac351d4a14b83c0934aa4ca75a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:30.618657 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8259e-c64c-4239-bb73-799194207b1b None admin] RBAC: Authorizing `identity:get_project(project_id=1bd0f7ac351d4a14b83c0934aa4ca75a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:30.620182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8259e-c64c-4239-bb73-799194207b1b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:30.621925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 70/136] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:30 2022] GET /identity/v3/projects/1bd0f7ac351d4a14b83c0934aa4ca75a => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:30.628530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0003a97-760f-41e5-92cb-a829da0474e6 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:30.692011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:30.786851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0003a97-760f-41e5-92cb-a829da0474e6 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:30.858944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc322b-1578-436d-bc9e-916b7aeb5b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:30.859542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc322b-1578-436d-bc9e-916b7aeb5b4c None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:30.859826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc322b-1578-436d-bc9e-916b7aeb5b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:30.860103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc322b-1578-436d-bc9e-916b7aeb5b4c None admin] PATH_INFO: `/v3/projects/1bd0f7ac351d4a14b83c0934aa4ca75a/users/99d2701b53de47caa5ad2c67fb1f0e64/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:30.862141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc322b-1578-436d-bc9e-916b7aeb5b4c None admin] RBAC: Authorizing `identity:create_grant(project_id=1bd0f7ac351d4a14b83c0934aa4ca75a, user_id=99d2701b53de47caa5ad2c67fb1f0e64, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:30.863604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc322b-1578-436d-bc9e-916b7aeb5b4c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:30.878257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 67/137] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:30 2022] PUT /identity/v3/projects/1bd0f7ac351d4a14b83c0934aa4ca75a/users/99d2701b53de47caa5ad2c67fb1f0e64/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:03:31.235977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864178-e086-4fef-9760-b443bc8cfec5 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:31.236667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864178-e086-4fef-9760-b443bc8cfec5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:31.236667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864178-e086-4fef-9760-b443bc8cfec5 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:31.238137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 71/138] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:31.251138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7363-5478-487b-bdae-4a817e944234 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:31.251421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7363-5478-487b-bdae-4a817e944234 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:31.251795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7363-5478-487b-bdae-4a817e944234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:31.272047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a6fc7363-5478-487b-bdae-4a817e944234 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:31.277429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a6fc7363-5478-487b-bdae-4a817e944234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:31.354252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 68/139] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:31.946053 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f864178-e086-4fef-9760-b443bc8cfec5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:32.080401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f864178-e086-4fef-9760-b443bc8cfec5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:32.087693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:32.134005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a82015-c485-4fca-a048-be1006fc6a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:32.134662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82015-c485-4fca-a048-be1006fc6a97 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:32.136183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bbf41-96a0-4c5b-a001-9e7536c92678 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:32.136674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bbf41-96a0-4c5b-a001-9e7536c92678 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:32.136854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bbf41-96a0-4c5b-a001-9e7536c92678 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:32.139939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 69/140] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:32.140109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82015-c485-4fca-a048-be1006fc6a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:32.140193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82015-c485-4fca-a048-be1006fc6a97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:32.141579 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a82015-c485-4fca-a048-be1006fc6a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:32.143284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a82015-c485-4fca-a048-be1006fc6a97 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:32.148972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 72/141] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:32.151226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aecb8b-1564-4863-982d-84dd840e3c9d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:32.151512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aecb8b-1564-4863-982d-84dd840e3c9d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:32.151785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aecb8b-1564-4863-982d-84dd840e3c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:32.159914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a82015-c485-4fca-a048-be1006fc6a97 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:32.189121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-89aecb8b-1564-4863-982d-84dd840e3c9d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:32.190386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-89aecb8b-1564-4863-982d-84dd840e3c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:32.259743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:32.351175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 70/142] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:32.371277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a82015-c485-4fca-a048-be1006fc6a97 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:32.409333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ce823-b442-4c58-8e1f-a0cf4ede668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:32.409942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ce823-b442-4c58-8e1f-a0cf4ede668e None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:32.410221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ce823-b442-4c58-8e1f-a0cf4ede668e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:32.410504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ce823-b442-4c58-8e1f-a0cf4ede668e None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:32.411158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ce823-b442-4c58-8e1f-a0cf4ede668e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:32.412692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ce823-b442-4c58-8e1f-a0cf4ede668e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:32.437124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 73/143] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:03:32 2022] POST /identity/v3/users => generated 283 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:32.710874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06303b7-473a-48ae-81e3-0bba59540227 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:32.711080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06303b7-473a-48ae-81e3-0bba59540227 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:32.711365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06303b7-473a-48ae-81e3-0bba59540227 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:32.712905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 71/144] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:32.719604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024240da-4953-4f7b-9f48-3c2f25ffb6c8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:32.719939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024240da-4953-4f7b-9f48-3c2f25ffb6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:32.720213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024240da-4953-4f7b-9f48-3c2f25ffb6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:32.744045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-024240da-4953-4f7b-9f48-3c2f25ffb6c8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:32.745277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-024240da-4953-4f7b-9f48-3c2f25ffb6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:32.780641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:32.834377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 74/145] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:32.859847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65930a6c-8068-4610-9c92-093e754c753b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:32.860211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65930a6c-8068-4610-9c92-093e754c753b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:32.860465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65930a6c-8068-4610-9c92-093e754c753b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:32.862127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 72/146] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:32.874281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821a22f-9342-49e3-9b5d-631f3974c7c4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:32.875042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821a22f-9342-49e3-9b5d-631f3974c7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:32.875042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821a22f-9342-49e3-9b5d-631f3974c7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:32.875798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65930a6c-8068-4610-9c92-093e754c753b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:32.905780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e821a22f-9342-49e3-9b5d-631f3974c7c4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:32.906989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e821a22f-9342-49e3-9b5d-631f3974c7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:33.037461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 75/147] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:33.072534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65930a6c-8068-4610-9c92-093e754c753b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:33.112156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25bf9-656f-408a-a620-2ef957cac9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:33.112826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25bf9-656f-408a-a620-2ef957cac9de None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.113110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25bf9-656f-408a-a620-2ef957cac9de None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.113385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25bf9-656f-408a-a620-2ef957cac9de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:33.124436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a25bf9-656f-408a-a620-2ef957cac9de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f5a25bf9-656f-408a-a620-2ef957cac9de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:33.129417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:33.130303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:33.131086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 73/148] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:03:33.140425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821a22f-9342-49e3-9b5d-631f3974c7c4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:33.264763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25bf9-656f-408a-a620-2ef957cac9de None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:33.279441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:33.289923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821a22f-9342-49e3-9b5d-631f3974c7c4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:33.306226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:33.331146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b4447-aa15-41fd-b5cf-135f011db544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:33.332004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b4447-aa15-41fd-b5cf-135f011db544 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.332452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b4447-aa15-41fd-b5cf-135f011db544 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.332932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b4447-aa15-41fd-b5cf-135f011db544 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:33.333803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b4447-aa15-41fd-b5cf-135f011db544 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:33.334267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b4447-aa15-41fd-b5cf-135f011db544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:33.335825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b4447-aa15-41fd-b5cf-135f011db544 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:33.346250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 76/149] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:33.352590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bafb7-c142-4551-940b-573e07170852 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.357051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bafb7-c142-4551-940b-573e07170852 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.357335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bafb7-c142-4551-940b-573e07170852 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:33.359092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 77/150] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:33.359881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bafb7-c142-4551-940b-573e07170852 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:33.409073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25bf9-656f-408a-a620-2ef957cac9de None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:33.457343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b448a-3266-4a5d-b9b5-007ca6940a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:33.458002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b448a-3266-4a5d-b9b5-007ca6940a1a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.458287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b448a-3266-4a5d-b9b5-007ca6940a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.458563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b448a-3266-4a5d-b9b5-007ca6940a1a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:33.472642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b448a-3266-4a5d-b9b5-007ca6940a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-475b448a-3266-4a5d-b9b5-007ca6940a1a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:33.477591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:33.478443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:33.479209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 74/151] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:03:33.479857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a65e-1a04-472b-bc65-310ffa00658b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.480690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a65e-1a04-472b-bc65-310ffa00658b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.480690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4a65e-1a04-472b-bc65-310ffa00658b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:33.501353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-08d4a65e-1a04-472b-bc65-310ffa00658b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:33.502573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-08d4a65e-1a04-472b-bc65-310ffa00658b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:33.523180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bafb7-c142-4551-940b-573e07170852 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:33.563400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c709-c3db-4ff6-8077-91e4f65754e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:33.564083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c709-c3db-4ff6-8077-91e4f65754e5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.564375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c709-c3db-4ff6-8077-91e4f65754e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.564781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c709-c3db-4ff6-8077-91e4f65754e5 None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:33.565947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76c709-c3db-4ff6-8077-91e4f65754e5 None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:33.568092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76c709-c3db-4ff6-8077-91e4f65754e5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:33.569823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 78/152] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:33 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:33.570656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c709-c3db-4ff6-8077-91e4f65754e5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:33.587511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 75/153] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:33.588380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4a65e-1a04-472b-bc65-310ffa00658b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:33.605409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:33.614350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:33.698994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4a65e-1a04-472b-bc65-310ffa00658b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:33.728902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c709-c3db-4ff6-8077-91e4f65754e5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:33.742282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:33.742907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.743380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.743744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:33.744429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:33.744762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:33.746175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:33.764491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 76/154] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:33.765556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:33.804077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef16bca-3bef-4615-a22a-abfb8df510e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:33.804781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef16bca-3bef-4615-a22a-abfb8df510e1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.805116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef16bca-3bef-4615-a22a-abfb8df510e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.805393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef16bca-3bef-4615-a22a-abfb8df510e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:33.833612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef16bca-3bef-4615-a22a-abfb8df510e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-9ef16bca-3bef-4615-a22a-abfb8df510e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:33.839509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:33.840395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:33.841963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:33.844485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 79/155] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:33.845527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef16bca-3bef-4615-a22a-abfb8df510e1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:33.912965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:33.925886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cdcd2-3a31-4747-8190-606740ac34b2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:33.983237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:33.984051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114add22-aa00-430b-88e5-8eb095b86832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:33.984719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114add22-aa00-430b-88e5-8eb095b86832 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:33.985031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114add22-aa00-430b-88e5-8eb095b86832 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:33.986482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114add22-aa00-430b-88e5-8eb095b86832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.010610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef16bca-3bef-4615-a22a-abfb8df510e1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:34.049543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065b7a8-5193-4244-b6af-c97859d67f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:34.050481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b7a8-5193-4244-b6af-c97859d67f82 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:34.050775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b7a8-5193-4244-b6af-c97859d67f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:34.051047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b7a8-5193-4244-b6af-c97859d67f82 None admin] PATH_INFO: `/v3/users/d2ba1cc0a455447ab883edd9eb275586` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.052286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065b7a8-5193-4244-b6af-c97859d67f82 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ba1cc0a455447ab883edd9eb275586)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:34.053805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065b7a8-5193-4244-b6af-c97859d67f82 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:34.055477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 80/156] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:33 2022] GET /identity/v3/users/d2ba1cc0a455447ab883edd9eb275586 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:34.056253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065b7a8-5193-4244-b6af-c97859d67f82 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:34.130897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114add22-aa00-430b-88e5-8eb095b86832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:34.135135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114add22-aa00-430b-88e5-8eb095b86832 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:34.245899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065b7a8-5193-4244-b6af-c97859d67f82 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:34.280938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:34.317540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:34.318149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:34.318426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:34.318987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.319091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 77/157] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:34.319358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:34.319656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:34.320049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5f5c4-694d-48c8-93f3-e880c675be5c None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:34.320342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5f5c4-694d-48c8-93f3-e880c675be5c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:34.320641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5f5c4-694d-48c8-93f3-e880c675be5c None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.321141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:34.322047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 78/158] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:34.322615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5f5c4-694d-48c8-93f3-e880c675be5c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:34.331665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 81/159] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:34.333212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:34.359379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:34.520770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300dcee-543a-40a1-8aab-cd9a9b897b65 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:34.526029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5f5c4-694d-48c8-93f3-e880c675be5c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:34.587915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b585a4-8276-494e-bf84-edee75eba745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:34.588606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b585a4-8276-494e-bf84-edee75eba745 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:34.588886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b585a4-8276-494e-bf84-edee75eba745 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:34.589153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b585a4-8276-494e-bf84-edee75eba745 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.590284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b585a4-8276-494e-bf84-edee75eba745 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:34.592511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b585a4-8276-494e-bf84-edee75eba745 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:34.595371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 79/160] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:34 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:34.595494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86569ef8-3fbb-4bf4-9cb3-2edcf5702b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:34.596014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569ef8-3fbb-4bf4-9cb3-2edcf5702b7e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:34.596221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569ef8-3fbb-4bf4-9cb3-2edcf5702b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:34.596364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b585a4-8276-494e-bf84-edee75eba745 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:34.596496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86569ef8-3fbb-4bf4-9cb3-2edcf5702b7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.625943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86569ef8-3fbb-4bf4-9cb3-2edcf5702b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-86569ef8-3fbb-4bf4-9cb3-2edcf5702b7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:34.631740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:34.632786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:34.633577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 82/161] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:03:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:34.634273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f863-8645-49ee-86d7-89154ef870c4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:34.634343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f863-8645-49ee-86d7-89154ef870c4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:34.635209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03f863-8645-49ee-86d7-89154ef870c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.675584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:34.678770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8c03f863-8645-49ee-86d7-89154ef870c4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:34.679995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8c03f863-8645-49ee-86d7-89154ef870c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:34.781083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:34.785566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 83/162] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:34.793135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03f863-8645-49ee-86d7-89154ef870c4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:34.811057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b585a4-8276-494e-bf84-edee75eba745 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:34.923041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3001f7-8241-4dcc-a093-50ce77746355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:34.923703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3001f7-8241-4dcc-a093-50ce77746355 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:34.923985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3001f7-8241-4dcc-a093-50ce77746355 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:34.924260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3001f7-8241-4dcc-a093-50ce77746355 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.925862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3001f7-8241-4dcc-a093-50ce77746355 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:34.927433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3001f7-8241-4dcc-a093-50ce77746355 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:34.929235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 80/163] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:34 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:34.936757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3001f7-8241-4dcc-a093-50ce77746355 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:34.937331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03f863-8645-49ee-86d7-89154ef870c4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:34.978136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9384b-c8b4-474e-84d4-a50d48025fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:34.984595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9384b-c8b4-474e-84d4-a50d48025fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:34.984595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9384b-c8b4-474e-84d4-a50d48025fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:34.984595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9384b-c8b4-474e-84d4-a50d48025fc9 None admin] PATH_INFO: `/v3/users/60fc2717779e40a0a55137c62461f753` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:34.984595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9384b-c8b4-474e-84d4-a50d48025fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=60fc2717779e40a0a55137c62461f753)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:34.984595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9384b-c8b4-474e-84d4-a50d48025fc9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:34.984595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 84/164] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:34 2022] GET /identity/v3/users/60fc2717779e40a0a55137c62461f753 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:34.985058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9384b-c8b4-474e-84d4-a50d48025fc9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:35.087014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:35.099717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9384b-c8b4-474e-84d4-a50d48025fc9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:35.108479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3001f7-8241-4dcc-a093-50ce77746355 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:35.126673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:35.142826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:35.143803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:35.144287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:35.144781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:35.145645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:35.146049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ce2c1-7191-4281-b334-3cb4437db919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:35.146395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:35.146720 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ce2c1-7191-4281-b334-3cb4437db919 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:35.147002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ce2c1-7191-4281-b334-3cb4437db919 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:35.147272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ce2c1-7191-4281-b334-3cb4437db919 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/users/d2ba1cc0a455447ab883edd9eb275586/roles/424ecb0deee84a068044178a8ee9ad52` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:35.148651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:35.149263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ce2c1-7191-4281-b334-3cb4437db919 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f379c8467d44bcbbf9cbc6aac3a3231, user_id=d2ba1cc0a455447ab883edd9eb275586, role_id=424ecb0deee84a068044178a8ee9ad52)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:35.150719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ce2c1-7191-4281-b334-3cb4437db919 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:35.159674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 85/165] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:03:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:35.160935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:35.180906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 81/166] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:34 2022] PUT /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/users/d2ba1cc0a455447ab883edd9eb275586/roles/424ecb0deee84a068044178a8ee9ad52 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:35.181768 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ce2c1-7191-4281-b334-3cb4437db919 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:35.288558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a6ff1-b3fd-45bd-bb10-da70986a92ea None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:35.294849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ce2c1-7191-4281-b334-3cb4437db919 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:35.334130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c85fbb-8888-4c0f-b9a3-850a3c673710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:35.334756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c85fbb-8888-4c0f-b9a3-850a3c673710 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:35.335040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c85fbb-8888-4c0f-b9a3-850a3c673710 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:35.335315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c85fbb-8888-4c0f-b9a3-850a3c673710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:35.340070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:35.356200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:35.358027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:35.358734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:35.359295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:35.360359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] RBAC: Adding query filter params (role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231, user.id=99d2701b53de47caa5ad2c67fb1f0e64) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:35.363362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:35.365215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:35.377219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 86/167] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:35 2022] GET /identity/v3/role_assignments?user.id=99d2701b53de47caa5ad2c67fb1f0e64&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:35.378673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:35.391236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:35.423867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c85fbb-8888-4c0f-b9a3-850a3c673710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:35.425625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c85fbb-8888-4c0f-b9a3-850a3c673710 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:35.538256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 82/168] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:35.539375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c85fbb-8888-4c0f-b9a3-850a3c673710 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:35.610739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30a806-719f-4a8d-819e-49e057667a0a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:35.647903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:35.679731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c85fbb-8888-4c0f-b9a3-850a3c673710 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:35.692841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77616f-8909-4051-9063-3e5684c6a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:35.693973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77616f-8909-4051-9063-3e5684c6a20c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:35.694452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77616f-8909-4051-9063-3e5684c6a20c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:35.694939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77616f-8909-4051-9063-3e5684c6a20c None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:35.697064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77616f-8909-4051-9063-3e5684c6a20c None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:35.698829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77616f-8909-4051-9063-3e5684c6a20c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:35.702184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 87/169] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:35 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:35.704080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77616f-8909-4051-9063-3e5684c6a20c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:35.722152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f06284-818a-4ecc-928a-6ecf04467402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:35.722808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f06284-818a-4ecc-928a-6ecf04467402 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:35.723092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f06284-818a-4ecc-928a-6ecf04467402 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:35.723367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f06284-818a-4ecc-928a-6ecf04467402 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:35.736269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f06284-818a-4ecc-928a-6ecf04467402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-e0f06284-818a-4ecc-928a-6ecf04467402 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:35.742329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:35.743888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 83/170] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:35.745186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f06284-818a-4ecc-928a-6ecf04467402 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:35.846562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:35.887631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f06284-818a-4ecc-928a-6ecf04467402 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:35.922717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77616f-8909-4051-9063-3e5684c6a20c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:35.958320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb453a-f394-4cc5-a104-17caf4514277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:35.958950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb453a-f394-4cc5-a104-17caf4514277 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:35.959232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb453a-f394-4cc5-a104-17caf4514277 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:35.959656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb453a-f394-4cc5-a104-17caf4514277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:35.964901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:36.001315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff1433-0505-4b44-aca5-9102fbd9de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:36.001966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1433-0505-4b44-aca5-9102fbd9de9d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.002248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1433-0505-4b44-aca5-9102fbd9de9d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.002563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff1433-0505-4b44-aca5-9102fbd9de9d None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.003758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff1433-0505-4b44-aca5-9102fbd9de9d None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:36.005259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff1433-0505-4b44-aca5-9102fbd9de9d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:36.011942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 88/171] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:35 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:36.012964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff1433-0505-4b44-aca5-9102fbd9de9d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:36.052920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb453a-f394-4cc5-a104-17caf4514277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:36.054562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb453a-f394-4cc5-a104-17caf4514277 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:36.190047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 84/172] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:36.195588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb453a-f394-4cc5-a104-17caf4514277 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:36.196454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff1433-0505-4b44-aca5-9102fbd9de9d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:36.234772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:36.242809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:36.243563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.244103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.244479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.245614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:36.246017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:36.248655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:36.255522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:36.259501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 89/173] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:36.260428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:36.348424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb453a-f394-4cc5-a104-17caf4514277 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:36.397141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:36.397764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.398047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.398323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.399068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231, user.id=60fc2717779e40a0a55137c62461f753) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:36.399378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:36.400917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:36.408323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 85/174] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:36 2022] GET /identity/v3/role_assignments?user.id=60fc2717779e40a0a55137c62461f753&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:36.409191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:36.474868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3996ed4-cc4b-40ff-b4d8-5b12c4aa1019 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:36.585377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a0e33-f049-4e0f-8273-8d1c22a559a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:36.586030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a0e33-f049-4e0f-8273-8d1c22a559a5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.586307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a0e33-f049-4e0f-8273-8d1c22a559a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.586575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a0e33-f049-4e0f-8273-8d1c22a559a5 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.587841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a0e33-f049-4e0f-8273-8d1c22a559a5 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:36.589315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a0e33-f049-4e0f-8273-8d1c22a559a5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:36.590948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 90/175] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:36 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:36.593779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c473ce-80e5-40bc-a083-4ddb063fd9c4 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.594061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c473ce-80e5-40bc-a083-4ddb063fd9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.594330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c473ce-80e5-40bc-a083-4ddb063fd9c4 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.599549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 91/176] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:36.601867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c473ce-80e5-40bc-a083-4ddb063fd9c4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:36.645459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:36.648510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58340346-02c7-40a5-900b-5ef474af577a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:36.718898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:36.722817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff4788-4cd2-4634-9a7f-dbda4986a8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:36.723473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff4788-4cd2-4634-9a7f-dbda4986a8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.723755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff4788-4cd2-4634-9a7f-dbda4986a8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.724119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff4788-4cd2-4634-9a7f-dbda4986a8b8 None admin] PATH_INFO: `/v3/users/99d2701b53de47caa5ad2c67fb1f0e64` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.726352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff4788-4cd2-4634-9a7f-dbda4986a8b8 None admin] RBAC: Authorizing `identity:get_user(user_id=99d2701b53de47caa5ad2c67fb1f0e64)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:36.729042 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff4788-4cd2-4634-9a7f-dbda4986a8b8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:36.731322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 86/177] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:36 2022] GET /identity/v3/users/99d2701b53de47caa5ad2c67fb1f0e64 => generated 275 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:36.733647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7c6df-03ea-4c05-9ad7-6102fc78dcc2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.734012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7c6df-03ea-4c05-9ad7-6102fc78dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.734286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7c6df-03ea-4c05-9ad7-6102fc78dcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.771797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-7fb7c6df-03ea-4c05-9ad7-6102fc78dcc2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:36.773036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7fb7c6df-03ea-4c05-9ad7-6102fc78dcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:36.806954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c473ce-80e5-40bc-a083-4ddb063fd9c4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:36.885943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:36.886575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.886851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.887116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.887984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] RBAC: Adding query filter params (role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231, user.id=99d2701b53de47caa5ad2c67fb1f0e64) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:36.889278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:36.893342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:36.901386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 92/178] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:36 2022] GET /identity/v3/role_assignments?user.id=99d2701b53de47caa5ad2c67fb1f0e64&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=79dbf286cf884619af62f33ea75c52bf => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:36.902309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:36.908599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 87/179] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:36.910594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413b40-1182-4ed9-9514-f1291770aa03 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.911121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413b40-1182-4ed9-9514-f1291770aa03 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.911615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413b40-1182-4ed9-9514-f1291770aa03 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.913307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 88/180] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:36.929165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4eade-83ff-4f6b-abb8-4e0f94fa0dce None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:36.929785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4eade-83ff-4f6b-abb8-4e0f94fa0dce None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:36.930260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4eade-83ff-4f6b-abb8-4e0f94fa0dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:36.975820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f8c4eade-83ff-4f6b-abb8-4e0f94fa0dce None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:36.977538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f8c4eade-83ff-4f6b-abb8-4e0f94fa0dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:37.035731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02c20c-4a28-4697-b1c9-5f73b66dc7ea None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:37.065316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:37.083076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4888428-5644-4e9c-a924-a72e2f0357d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:37.083710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4888428-5644-4e9c-a924-a72e2f0357d0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:37.083987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4888428-5644-4e9c-a924-a72e2f0357d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:37.084261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4888428-5644-4e9c-a924-a72e2f0357d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:37.148593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 89/181] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:37.181463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4888428-5644-4e9c-a924-a72e2f0357d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:37.183264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4888428-5644-4e9c-a924-a72e2f0357d0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:37.301508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 93/182] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:37.307318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4eade-83ff-4f6b-abb8-4e0f94fa0dce None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:37.350076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:37.368108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4888428-5644-4e9c-a924-a72e2f0357d0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:37.423395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:37.479556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4eade-83ff-4f6b-abb8-4e0f94fa0dce None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:37.546807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa723f-7a83-49ca-9a97-4d50d0e92e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:37.547815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa723f-7a83-49ca-9a97-4d50d0e92e4e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:37.548365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa723f-7a83-49ca-9a97-4d50d0e92e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:37.548695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa723f-7a83-49ca-9a97-4d50d0e92e4e None admin] PATH_INFO: `/v3/projects/1bd0f7ac351d4a14b83c0934aa4ca75a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:37.550034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa723f-7a83-49ca-9a97-4d50d0e92e4e None admin] RBAC: Authorizing `identity:get_project(project_id=1bd0f7ac351d4a14b83c0934aa4ca75a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:37.552460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa723f-7a83-49ca-9a97-4d50d0e92e4e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:37.554633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 90/183] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:37 2022] GET /identity/v3/projects/1bd0f7ac351d4a14b83c0934aa4ca75a => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:37.557271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573ce2-44c0-47f8-aeb8-3e6a7a01f338 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:37.557569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573ce2-44c0-47f8-aeb8-3e6a7a01f338 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:37.557846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573ce2-44c0-47f8-aeb8-3e6a7a01f338 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:37.559292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4888428-5644-4e9c-a924-a72e2f0357d0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:37.559516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 91/184] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:37.565419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573ce2-44c0-47f8-aeb8-3e6a7a01f338 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:37.613886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0da05c-c6fd-45ab-96ee-94c1502ed972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:37.614900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da05c-c6fd-45ab-96ee-94c1502ed972 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:37.615281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da05c-c6fd-45ab-96ee-94c1502ed972 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:37.615559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da05c-c6fd-45ab-96ee-94c1502ed972 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:37.627782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0da05c-c6fd-45ab-96ee-94c1502ed972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5c0da05c-c6fd-45ab-96ee-94c1502ed972 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:37.634970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:37.642614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 94/185] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:37.644311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe796-7ade-4b93-8706-8bb7662315ce None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:37.645522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe796-7ade-4b93-8706-8bb7662315ce None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:37.645522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbe796-7ade-4b93-8706-8bb7662315ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:37.658476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c8fbe796-7ade-4b93-8706-8bb7662315ce None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:37.659679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c8fbe796-7ade-4b93-8706-8bb7662315ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:37.699318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:37.708659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573ce2-44c0-47f8-aeb8-3e6a7a01f338 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:37.746156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 95/186] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:37.757172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbe796-7ade-4b93-8706-8bb7662315ce None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:37.760299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:37.761326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:37.761866 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:37.762145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:37.763170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] RBAC: Adding query filter params (role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=1bd0f7ac351d4a14b83c0934aa4ca75a, user.id=99d2701b53de47caa5ad2c67fb1f0e64) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:37.763865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:37.765146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:37.767171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:37.775819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 92/187] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:37 2022] GET /identity/v3/role_assignments?user.id=99d2701b53de47caa5ad2c67fb1f0e64&scope.project.id=1bd0f7ac351d4a14b83c0934aa4ca75a&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:37.777049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:37.947231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbe796-7ade-4b93-8706-8bb7662315ce None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:37.974527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895eb0ac-011f-40be-b500-9a67b17de45f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:37.988295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edaf8de-a720-4b28-8200-7737817878dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:37.989382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaf8de-a720-4b28-8200-7737817878dc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:37.989664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaf8de-a720-4b28-8200-7737817878dc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:37.989939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaf8de-a720-4b28-8200-7737817878dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.002297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaf8de-a720-4b28-8200-7737817878dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-9edaf8de-a720-4b28-8200-7737817878dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:38.007628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:38.008521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 96/188] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:03:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:38.012440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b381df1-010d-4c2b-8b17-ed8a4a77d2eb None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.013734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b381df1-010d-4c2b-8b17-ed8a4a77d2eb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.013734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b381df1-010d-4c2b-8b17-ed8a4a77d2eb None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.013734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 97/189] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:38.019715 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f964e2-7a6d-4f65-9b0c-63a08b35a2c9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.019995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f964e2-7a6d-4f65-9b0c-63a08b35a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.020292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f964e2-7a6d-4f65-9b0c-63a08b35a2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.025178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.026036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.026745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.027230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.029788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:38.030352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:38.032285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:38.041256 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a0f964e2-7a6d-4f65-9b0c-63a08b35a2c9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:38.041784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 93/190] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:38.042517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a0f964e2-7a6d-4f65-9b0c-63a08b35a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:38.042684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:38.076357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:38.115637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 98/191] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:38.116486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f964e2-7a6d-4f65-9b0c-63a08b35a2c9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:38.119494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:38.173942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6e06f-9328-4885-83fa-edfb5b29932f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:38.210609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.211133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.211412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.211686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.212387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:38.212729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:38.214396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:38.228260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 94/192] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:03:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:38.233713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:38.254191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f964e2-7a6d-4f65-9b0c-63a08b35a2c9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:38.292059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f80fee-3cf3-4af1-802c-6cfff07ba8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.292763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f80fee-3cf3-4af1-802c-6cfff07ba8ff None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.293137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f80fee-3cf3-4af1-802c-6cfff07ba8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.293409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f80fee-3cf3-4af1-802c-6cfff07ba8ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.295298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f80fee-3cf3-4af1-802c-6cfff07ba8ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:38.296866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f80fee-3cf3-4af1-802c-6cfff07ba8ff None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:38.300674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 99/193] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:38.300674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f80fee-3cf3-4af1-802c-6cfff07ba8ff None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:38.326670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:38.344883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec1933-5161-4735-a956-e4f9b5008d87 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:38.383624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65597bf-57fe-49f7-a71d-9a29baf07f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.384254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65597bf-57fe-49f7-a71d-9a29baf07f3f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.384529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65597bf-57fe-49f7-a71d-9a29baf07f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.384835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65597bf-57fe-49f7-a71d-9a29baf07f3f None admin] PATH_INFO: `/v3/users/7b5dedda01774f71929a65c06a137247` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.391998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:38.399809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65597bf-57fe-49f7-a71d-9a29baf07f3f None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dedda01774f71929a65c06a137247)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:38.401328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65597bf-57fe-49f7-a71d-9a29baf07f3f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:38.402992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 95/194] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:38 2022] GET /identity/v3/users/7b5dedda01774f71929a65c06a137247 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:38.404381 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65597bf-57fe-49f7-a71d-9a29baf07f3f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:38.425570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f80fee-3cf3-4af1-802c-6cfff07ba8ff None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:38.459909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620b3aa-12c4-4817-9cb0-b7e5d8a22f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.460737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620b3aa-12c4-4817-9cb0-b7e5d8a22f9e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.460954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620b3aa-12c4-4817-9cb0-b7e5d8a22f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.461228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620b3aa-12c4-4817-9cb0-b7e5d8a22f9e None admin] PATH_INFO: `/v3/users/7b5dedda01774f71929a65c06a137247` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.462367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620b3aa-12c4-4817-9cb0-b7e5d8a22f9e None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dedda01774f71929a65c06a137247)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:38.463846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620b3aa-12c4-4817-9cb0-b7e5d8a22f9e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:38.465623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 100/195] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:38 2022] GET /identity/v3/users/7b5dedda01774f71929a65c06a137247 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:38.466957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620b3aa-12c4-4817-9cb0-b7e5d8a22f9e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:38.509976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65597bf-57fe-49f7-a71d-9a29baf07f3f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:38.546080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b046c6-5e17-4d1f-b9d5-ffa44c94e0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.546685 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b046c6-5e17-4d1f-b9d5-ffa44c94e0db None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.546960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b046c6-5e17-4d1f-b9d5-ffa44c94e0db None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.547227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b046c6-5e17-4d1f-b9d5-ffa44c94e0db None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.547869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b046c6-5e17-4d1f-b9d5-ffa44c94e0db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:38.549359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b046c6-5e17-4d1f-b9d5-ffa44c94e0db None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:38.568168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:38.580609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 96/196] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:03:38 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:38.580609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b046c6-5e17-4d1f-b9d5-ffa44c94e0db None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:38.594440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620b3aa-12c4-4817-9cb0-b7e5d8a22f9e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:38.648170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5541c-8da8-45af-a1ca-3555686b7371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.648969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5541c-8da8-45af-a1ca-3555686b7371 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.649318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5541c-8da8-45af-a1ca-3555686b7371 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.649596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5541c-8da8-45af-a1ca-3555686b7371 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.659237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:38.662820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5541c-8da8-45af-a1ca-3555686b7371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5bc5541c-8da8-45af-a1ca-3555686b7371 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:38.668001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:38.669543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 101/197] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:38.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5541c-8da8-45af-a1ca-3555686b7371 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:38.718836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b046c6-5e17-4d1f-b9d5-ffa44c94e0db None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:38.768193 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b0c75-87ab-4819-9a45-605baa6f5829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.768856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b0c75-87ab-4819-9a45-605baa6f5829 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.769143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b0c75-87ab-4819-9a45-605baa6f5829 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.769427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b0c75-87ab-4819-9a45-605baa6f5829 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.803897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5541c-8da8-45af-a1ca-3555686b7371 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:38.849518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdeba33-bf4b-4c20-bd22-82e52e8e3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:38.850274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeba33-bf4b-4c20-bd22-82e52e8e3186 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:38.850618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeba33-bf4b-4c20-bd22-82e52e8e3186 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:38.850956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdeba33-bf4b-4c20-bd22-82e52e8e3186 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:38.868311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:38.873073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b0c75-87ab-4819-9a45-605baa6f5829 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:38.875121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b0c75-87ab-4819-9a45-605baa6f5829 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:38.954262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:38.958159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdeba33-bf4b-4c20-bd22-82e52e8e3186 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:38.959694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdeba33-bf4b-4c20-bd22-82e52e8e3186 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:39.002721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 97/198] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:39.003383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b0c75-87ab-4819-9a45-605baa6f5829 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:39.088364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 102/199] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:39.093558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdeba33-bf4b-4c20-bd22-82e52e8e3186 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:39.132892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:39.140817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b0c75-87ab-4819-9a45-605baa6f5829 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:39.186925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:39.187785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:39.188358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:39.188811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:39.189733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:39.190180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:39.191719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:39.202224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 98/200] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:39.204613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:39.210196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:39.213627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdeba33-bf4b-4c20-bd22-82e52e8e3186 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:39.259599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9a79-9bec-49cc-93a8-e5f81ed43d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:39.260719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9a79-9bec-49cc-93a8-e5f81ed43d16 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:39.261270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9a79-9bec-49cc-93a8-e5f81ed43d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:39.261723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9a79-9bec-49cc-93a8-e5f81ed43d16 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:39.264373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea9a79-9bec-49cc-93a8-e5f81ed43d16 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:39.267489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea9a79-9bec-49cc-93a8-e5f81ed43d16 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:39.270405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 103/201] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:39 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:39.271721 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9a79-9bec-49cc-93a8-e5f81ed43d16 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:39.351924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d7aba-8f2d-4ee5-ae5a-148ceba7510c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:39.387261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad0bca-c5e8-47c2-84a3-36a7024392eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:39.388435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0bca-c5e8-47c2-84a3-36a7024392eb None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:39.388749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0bca-c5e8-47c2-84a3-36a7024392eb None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:39.389110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad0bca-c5e8-47c2-84a3-36a7024392eb None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:39.390444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad0bca-c5e8-47c2-84a3-36a7024392eb None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:39.391036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9a79-9bec-49cc-93a8-e5f81ed43d16 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:39.391923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad0bca-c5e8-47c2-84a3-36a7024392eb None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:39.393610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 99/202] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:39 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:39.399351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad0bca-c5e8-47c2-84a3-36a7024392eb None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:39.442817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30adeca-01f8-4b17-9ec3-717b211b34c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:39.443454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30adeca-01f8-4b17-9ec3-717b211b34c2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:39.443733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30adeca-01f8-4b17-9ec3-717b211b34c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:39.444007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30adeca-01f8-4b17-9ec3-717b211b34c2 None admin] PATH_INFO: `/v3/users/d2ba1cc0a455447ab883edd9eb275586` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:39.445873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:39.446095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30adeca-01f8-4b17-9ec3-717b211b34c2 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ba1cc0a455447ab883edd9eb275586)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:39.450259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30adeca-01f8-4b17-9ec3-717b211b34c2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:39.456702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 104/203] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:39 2022] GET /identity/v3/users/d2ba1cc0a455447ab883edd9eb275586 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:39.457761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30adeca-01f8-4b17-9ec3-717b211b34c2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:39.499599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:39.539235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad0bca-c5e8-47c2-84a3-36a7024392eb None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:39.573034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30adeca-01f8-4b17-9ec3-717b211b34c2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:39.580154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:39.581379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:39.582401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:39.583254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:39.584498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=362cde2d45474fbcadbd8b85adbbac97, user.id=7b5dedda01774f71929a65c06a137247) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:39.585271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:39.587125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:39.601144 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 100/204] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:39 2022] GET /identity/v3/role_assignments?user.id=7b5dedda01774f71929a65c06a137247&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:39.602481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:39.612894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:39.619105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:39.619105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:39.619105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:39.619105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] RBAC: Adding query filter params (role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=362cde2d45474fbcadbd8b85adbbac97, user.id=7b5dedda01774f71929a65c06a137247) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:39.619105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:39.619105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:39.624655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 105/205] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:39 2022] GET /identity/v3/role_assignments?user.id=7b5dedda01774f71929a65c06a137247&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=79dbf286cf884619af62f33ea75c52bf => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:39.765985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:39.775694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9d6a7-8b3a-424e-9b18-6a6201a632b2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:39.816409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd47fd-c445-425c-93f6-d7e1074d8eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:39.816986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd47fd-c445-425c-93f6-d7e1074d8eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:39.817264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd47fd-c445-425c-93f6-d7e1074d8eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:39.817531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd47fd-c445-425c-93f6-d7e1074d8eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:39.897489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd47fd-c445-425c-93f6-d7e1074d8eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:39.899053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd47fd-c445-425c-93f6-d7e1074d8eb2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:40.000956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:40.010417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 101/206] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:40.017080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:40.129208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f5fbf-da87-4066-8c8f-c7528f1c1863 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:40.132956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:40.165837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b726b1-4929-4b23-a1cd-6e84dcfb3928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:40.166480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b726b1-4929-4b23-a1cd-6e84dcfb3928 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:40.166756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b726b1-4929-4b23-a1cd-6e84dcfb3928 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:40.167025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b726b1-4929-4b23-a1cd-6e84dcfb3928 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:40.168211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b726b1-4929-4b23-a1cd-6e84dcfb3928 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:40.171151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b726b1-4929-4b23-a1cd-6e84dcfb3928 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:40.173596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 106/207] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:40 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:40.180459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd47fd-c445-425c-93f6-d7e1074d8eb2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:40.311241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd47fd-c445-425c-93f6-d7e1074d8eb2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:40.348183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1809d6-8995-441e-9cd4-239e3e4536c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:40.348818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1809d6-8995-441e-9cd4-239e3e4536c8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:40.349094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1809d6-8995-441e-9cd4-239e3e4536c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:40.349362 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1809d6-8995-441e-9cd4-239e3e4536c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:40.350160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1809d6-8995-441e-9cd4-239e3e4536c8 None admin] RBAC: Adding query filter params (role.id=424ecb0deee84a068044178a8ee9ad52, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231, user.id=d2ba1cc0a455447ab883edd9eb275586) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:40.350469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1809d6-8995-441e-9cd4-239e3e4536c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:40.351930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1809d6-8995-441e-9cd4-239e3e4536c8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:40.358946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 102/208] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:40 2022] GET /identity/v3/role_assignments?user.id=d2ba1cc0a455447ab883edd9eb275586&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=424ecb0deee84a068044178a8ee9ad52 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:40.900408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba114-f16e-423c-baed-ce30d755d358 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:40.900836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba114-f16e-423c-baed-ce30d755d358 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:40.901012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba114-f16e-423c-baed-ce30d755d358 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:40.902538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 107/209] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:40.909164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953370a-fb28-4add-880e-c3fad199562a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:40.909705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953370a-fb28-4add-880e-c3fad199562a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:40.909782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953370a-fb28-4add-880e-c3fad199562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:40.931485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-2953370a-fb28-4add-880e-c3fad199562a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:40.932748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2953370a-fb28-4add-880e-c3fad199562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:40.991392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:41.011990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 103/210] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:41.065056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fc82-7b9a-49e6-bfa0-60bbe7767d24 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.065722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fc82-7b9a-49e6-bfa0-60bbe7767d24 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.065722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fc82-7b9a-49e6-bfa0-60bbe7767d24 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.067225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 108/211] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:41.074627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735c7a8-904d-4e5a-9fb3-a0981acdd565 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.075062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735c7a8-904d-4e5a-9fb3-a0981acdd565 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.075387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735c7a8-904d-4e5a-9fb3-a0981acdd565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.096068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4735c7a8-904d-4e5a-9fb3-a0981acdd565 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:41.097425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4735c7a8-904d-4e5a-9fb3-a0981acdd565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:41.180563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 104/212] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:41.311869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833fc82-7b9a-49e6-bfa0-60bbe7767d24 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:41.424046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833fc82-7b9a-49e6-bfa0-60bbe7767d24 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:41.463454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7d4d2-1d8a-4752-aaf2-9079c7501f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:41.464099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7d4d2-1d8a-4752-aaf2-9079c7501f40 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.464380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7d4d2-1d8a-4752-aaf2-9079c7501f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.464681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7d4d2-1d8a-4752-aaf2-9079c7501f40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.477935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7d4d2-1d8a-4752-aaf2-9079c7501f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:41.479805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735c7a8-904d-4e5a-9fb3-a0981acdd565 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f3a7d4d2-1d8a-4752-aaf2-9079c7501f40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:41.489093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:41.490016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:41.490825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 109/213] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:41.498126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7d4d2-1d8a-4752-aaf2-9079c7501f40 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:41.529959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:41.616960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735c7a8-904d-4e5a-9fb3-a0981acdd565 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:41.619757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7d4d2-1d8a-4752-aaf2-9079c7501f40 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:41.642903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:41.664389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:41.665319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.665794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.666231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.667079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:41.667547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:41.669887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:41.671122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f603-4431-4d0d-9ead-03adb25fd1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:41.672070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f603-4431-4d0d-9ead-03adb25fd1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.672360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f603-4431-4d0d-9ead-03adb25fd1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.672674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f603-4431-4d0d-9ead-03adb25fd1c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.674285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598f603-4431-4d0d-9ead-03adb25fd1c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:41.676010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598f603-4431-4d0d-9ead-03adb25fd1c5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:41.683592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 105/214] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:41.683761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 110/215] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:41.689921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f603-4431-4d0d-9ead-03adb25fd1c5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:41.690075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:41.810252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:41.824128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8438548-41f4-4e9e-9689-9ee710c78145 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:41.832018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f603-4431-4d0d-9ead-03adb25fd1c5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:41.867006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d89d2-c676-41eb-9699-1e24e2a1000a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:41.867575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d89d2-c676-41eb-9699-1e24e2a1000a None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.867884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d89d2-c676-41eb-9699-1e24e2a1000a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.868193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d89d2-c676-41eb-9699-1e24e2a1000a None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.868813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d89d2-c676-41eb-9699-1e24e2a1000a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:41.870302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d89d2-c676-41eb-9699-1e24e2a1000a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:41.881762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6a26e-c60e-43dc-b022-e7e8453b3fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:41.882744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6a26e-c60e-43dc-b022-e7e8453b3fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.883255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6a26e-c60e-43dc-b022-e7e8453b3fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.883729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6a26e-c60e-43dc-b022-e7e8453b3fd6 None admin] PATH_INFO: `/v3/users/24f7a30de104416e9a1f9c3a32e12b63` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.894455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 111/216] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:03:41 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:41.895714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b25f60-df41-4178-a7d7-2097fce7c76c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.896003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b25f60-df41-4178-a7d7-2097fce7c76c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.896450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b25f60-df41-4178-a7d7-2097fce7c76c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.897711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 112/217] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:41.908131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875217e-21de-4fc4-9f5f-63d3de2dd3b9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:41.908452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875217e-21de-4fc4-9f5f-63d3de2dd3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:41.908793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875217e-21de-4fc4-9f5f-63d3de2dd3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:41.912949 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6a26e-c60e-43dc-b022-e7e8453b3fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=24f7a30de104416e9a1f9c3a32e12b63)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:41.915472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6a26e-c60e-43dc-b022-e7e8453b3fd6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:41.919321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 106/218] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:41 2022] GET /identity/v3/users/24f7a30de104416e9a1f9c3a32e12b63 => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:41.930664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6a26e-c60e-43dc-b022-e7e8453b3fd6 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:41.936720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c875217e-21de-4fc4-9f5f-63d3de2dd3b9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:41.938002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c875217e-21de-4fc4-9f5f-63d3de2dd3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:42.000222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:42.044898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 113/219] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:42.077752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ea29-a551-486d-b572-d62cec38f6b4 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:42.078355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ea29-a551-486d-b572-d62cec38f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:42.078355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ea29-a551-486d-b572-d62cec38f6b4 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:42.079562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 114/220] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:03:42.084848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8fb4e-eccf-4f6c-a9bd-7ee8db136e6a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:42.085133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8fb4e-eccf-4f6c-a9bd-7ee8db136e6a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:42.085395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6a26e-c60e-43dc-b022-e7e8453b3fd6 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:42.085535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8fb4e-eccf-4f6c-a9bd-7ee8db136e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:42.107769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e0f8fb4e-eccf-4f6c-a9bd-7ee8db136e6a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:42.110141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e0f8fb4e-eccf-4f6c-a9bd-7ee8db136e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:42.142015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f00d92-0be8-4019-a3f7-60caede5b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:42.142015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f00d92-0be8-4019-a3f7-60caede5b672 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:42.142015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f00d92-0be8-4019-a3f7-60caede5b672 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:42.142015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f00d92-0be8-4019-a3f7-60caede5b672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:42.221962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 115/221] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:42.240173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f00d92-0be8-4019-a3f7-60caede5b672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:42.241736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f00d92-0be8-4019-a3f7-60caede5b672 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:42.303115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:42.335906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8fb4e-eccf-4f6c-a9bd-7ee8db136e6a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:42.347225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:42.382901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 107/222] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:42.390682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f00d92-0be8-4019-a3f7-60caede5b672 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:42.466616 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8fb4e-eccf-4f6c-a9bd-7ee8db136e6a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:42.503757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0afd2a-6173-42ba-aa40-ee06fe1c6eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:42.504389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0afd2a-6173-42ba-aa40-ee06fe1c6eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:42.504693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0afd2a-6173-42ba-aa40-ee06fe1c6eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:42.504996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0afd2a-6173-42ba-aa40-ee06fe1c6eb5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:42.517765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f00d92-0be8-4019-a3f7-60caede5b672 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:42.519592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0afd2a-6173-42ba-aa40-ee06fe1c6eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8b0afd2a-6173-42ba-aa40-ee06fe1c6eb5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:42.531861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:42.532820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:42.533614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 116/223] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:03:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:42.538966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0afd2a-6173-42ba-aa40-ee06fe1c6eb5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:42.566800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6061a34-37f4-470c-815a-47cc6701afa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:42.567977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6061a34-37f4-470c-815a-47cc6701afa8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:42.568407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6061a34-37f4-470c-815a-47cc6701afa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:42.569146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6061a34-37f4-470c-815a-47cc6701afa8 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:42.570870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6061a34-37f4-470c-815a-47cc6701afa8 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:42.572626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6061a34-37f4-470c-815a-47cc6701afa8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:42.575634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 108/224] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:42 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:42.577150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6061a34-37f4-470c-815a-47cc6701afa8 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:42.595352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:42.649330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:42.701344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0afd2a-6173-42ba-aa40-ee06fe1c6eb5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:42.709802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6061a34-37f4-470c-815a-47cc6701afa8 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:42.757810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:42.758599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:42.758599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:42.758599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:42.760605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:42.760605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:42.763601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:42.766949 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e259d1-17ea-4b14-a1e3-b7312e81f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:42.767596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e259d1-17ea-4b14-a1e3-b7312e81f7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:42.767882 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e259d1-17ea-4b14-a1e3-b7312e81f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:42.768153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e259d1-17ea-4b14-a1e3-b7312e81f7d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:42.783033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 117/225] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:03:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:42.784004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:42.785543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e259d1-17ea-4b14-a1e3-b7312e81f7d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-c0e259d1-17ea-4b14-a1e3-b7312e81f7d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:42.792554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:42.793493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:42.794890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:42.798076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 109/226] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:42.799367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e259d1-17ea-4b14-a1e3-b7312e81f7d1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:42.913724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d33b3-85a1-4c89-a976-8a754af344d8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:42.914682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:42.956812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e259d1-17ea-4b14-a1e3-b7312e81f7d1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:42.959880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:42.969935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:42.970944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:42.970944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:42.971411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:42.972238 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] RBAC: Adding query filter params (role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=362cde2d45474fbcadbd8b85adbbac97, user.id=24f7a30de104416e9a1f9c3a32e12b63) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:42.972759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:42.974439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:42.980229 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 118/227] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:42 2022] GET /identity/v3/role_assignments?user.id=24f7a30de104416e9a1f9c3a32e12b63&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:42.981305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:43.008623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dd129-8ec9-44f9-ad52-9834039faec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:43.009683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd129-8ec9-44f9-ad52-9834039faec1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:43.010116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd129-8ec9-44f9-ad52-9834039faec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:43.010742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd129-8ec9-44f9-ad52-9834039faec1 None admin] PATH_INFO: `/v3/users/7b5dedda01774f71929a65c06a137247` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:43.013906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dd129-8ec9-44f9-ad52-9834039faec1 None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dedda01774f71929a65c06a137247)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:43.017001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dd129-8ec9-44f9-ad52-9834039faec1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:43.019316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 110/228] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:42 2022] GET /identity/v3/users/7b5dedda01774f71929a65c06a137247 => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:43.030797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dd129-8ec9-44f9-ad52-9834039faec1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:43.112603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436bd58-2501-4bf3-9916-90b6ea0f5865 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:43.160551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:43.161581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:43.162061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:43.162791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:43.164407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:43.165263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:43.167166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:43.184042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 119/229] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:43.190520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:43.231463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dd129-8ec9-44f9-ad52-9834039faec1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:43.289984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d717fc2-af60-4f0f-8bbd-2cfcf79d5556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:43.290988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d717fc2-af60-4f0f-8bbd-2cfcf79d5556 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:43.291462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d717fc2-af60-4f0f-8bbd-2cfcf79d5556 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:43.291927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d717fc2-af60-4f0f-8bbd-2cfcf79d5556 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:43.293473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d717fc2-af60-4f0f-8bbd-2cfcf79d5556 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:43.296023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d717fc2-af60-4f0f-8bbd-2cfcf79d5556 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:43.302088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 111/230] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:43 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:43.306409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d717fc2-af60-4f0f-8bbd-2cfcf79d5556 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:43.316968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:43.322864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:43.404302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9398cb-f888-49c4-b7e7-93948ba0af6e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:43.462102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969dbfc-85b2-4ebc-8867-360bc0160044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:43.462620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969dbfc-85b2-4ebc-8867-360bc0160044 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:43.462899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969dbfc-85b2-4ebc-8867-360bc0160044 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:43.463171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969dbfc-85b2-4ebc-8867-360bc0160044 None admin] PATH_INFO: `/v3/users/7b5dedda01774f71929a65c06a137247` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:43.465617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969dbfc-85b2-4ebc-8867-360bc0160044 None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dedda01774f71929a65c06a137247)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:43.467727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969dbfc-85b2-4ebc-8867-360bc0160044 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:43.470550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 120/231] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:43 2022] GET /identity/v3/users/7b5dedda01774f71929a65c06a137247 => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:43.475836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969dbfc-85b2-4ebc-8867-360bc0160044 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:43.498034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d717fc2-af60-4f0f-8bbd-2cfcf79d5556 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:43.539508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf38689-c23e-471e-b606-7463a9024c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:43.540206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf38689-c23e-471e-b606-7463a9024c6d None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:43.540492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf38689-c23e-471e-b606-7463a9024c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:43.540801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf38689-c23e-471e-b606-7463a9024c6d None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97/users/7b5dedda01774f71929a65c06a137247/roles/79dbf286cf884619af62f33ea75c52bf` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:43.543123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf38689-c23e-471e-b606-7463a9024c6d None admin] RBAC: Authorizing `identity:create_grant(project_id=362cde2d45474fbcadbd8b85adbbac97, user_id=7b5dedda01774f71929a65c06a137247, role_id=79dbf286cf884619af62f33ea75c52bf)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:43.545831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf38689-c23e-471e-b606-7463a9024c6d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:43.570501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 112/232] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:43 2022] PUT /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97/users/7b5dedda01774f71929a65c06a137247/roles/79dbf286cf884619af62f33ea75c52bf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:43.645638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969dbfc-85b2-4ebc-8867-360bc0160044 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:43.678862 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:43.692299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd7e40-7bd1-461c-82ba-5084e3385361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:43.692972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7e40-7bd1-461c-82ba-5084e3385361 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:43.693451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7e40-7bd1-461c-82ba-5084e3385361 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:43.693451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7e40-7bd1-461c-82ba-5084e3385361 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:43.694865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd7e40-7bd1-461c-82ba-5084e3385361 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:43.696820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd7e40-7bd1-461c-82ba-5084e3385361 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:43.699399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 121/233] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:43 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:43.708409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf38689-c23e-471e-b606-7463a9024c6d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:43.816202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:43.861220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf38689-c23e-471e-b606-7463a9024c6d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:43.899662 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f30cf-5096-44d9-ab9c-290de79e454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:43.900347 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f30cf-5096-44d9-ab9c-290de79e454c None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:43.900652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f30cf-5096-44d9-ab9c-290de79e454c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:43.900932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f30cf-5096-44d9-ab9c-290de79e454c None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97/users/7b5dedda01774f71929a65c06a137247/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:43.902888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f30cf-5096-44d9-ab9c-290de79e454c None admin] RBAC: Authorizing `identity:create_grant(project_id=362cde2d45474fbcadbd8b85adbbac97, user_id=7b5dedda01774f71929a65c06a137247, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:43.905409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f30cf-5096-44d9-ab9c-290de79e454c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:43.924057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 113/234] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:43 2022] PUT /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97/users/7b5dedda01774f71929a65c06a137247/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:44.658933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786508-69be-44ee-a060-58055549b0ad None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:44.659459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786508-69be-44ee-a060-58055549b0ad None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:44.659459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786508-69be-44ee-a060-58055549b0ad None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:44.661067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 122/235] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:44.666340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16a21-61bc-4082-94b2-8138194ce267 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:44.666636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16a21-61bc-4082-94b2-8138194ce267 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:44.666918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16a21-61bc-4082-94b2-8138194ce267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:44.688661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f5a16a21-61bc-4082-94b2-8138194ce267 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:44.689916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f5a16a21-61bc-4082-94b2-8138194ce267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:44.771045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 114/236] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:44.791100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b846b2-661b-40af-adbd-25caba1f32c6 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:44.791396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b846b2-661b-40af-adbd-25caba1f32c6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:44.791663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b846b2-661b-40af-adbd-25caba1f32c6 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:44.793213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 123/237] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:44.798596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a9e68-0c5e-4885-8d1d-52d1e09db8c4 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:44.798890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a9e68-0c5e-4885-8d1d-52d1e09db8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:44.799163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a9e68-0c5e-4885-8d1d-52d1e09db8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:44.819433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-1d1a9e68-0c5e-4885-8d1d-52d1e09db8c4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:44.820672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1d1a9e68-0c5e-4885-8d1d-52d1e09db8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:44.879526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:44.902644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 115/238] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:45.190677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b846b2-661b-40af-adbd-25caba1f32c6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:45.239683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a9e68-0c5e-4885-8d1d-52d1e09db8c4 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:45.397309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b846b2-661b-40af-adbd-25caba1f32c6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:45.420237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a9e68-0c5e-4885-8d1d-52d1e09db8c4 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:45.443166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99d769-7adb-40b9-aa85-0a91cfa3ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:45.444104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99d769-7adb-40b9-aa85-0a91cfa3ad54 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:45.444391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99d769-7adb-40b9-aa85-0a91cfa3ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:45.444699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99d769-7adb-40b9-aa85-0a91cfa3ad54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:45.445954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99d769-7adb-40b9-aa85-0a91cfa3ad54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:45.448216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99d769-7adb-40b9-aa85-0a91cfa3ad54 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:45.456410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 124/239] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:45.458333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571d2da-1a09-46b2-a688-54025d4ff783 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:45.458718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571d2da-1a09-46b2-a688-54025d4ff783 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:45.458934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571d2da-1a09-46b2-a688-54025d4ff783 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:45.460725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 125/240] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:45.462467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571d2da-1a09-46b2-a688-54025d4ff783 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:45.476553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd250d-9295-4e9c-bf69-89dd3fb4b448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:45.477221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd250d-9295-4e9c-bf69-89dd3fb4b448 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:45.477499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd250d-9295-4e9c-bf69-89dd3fb4b448 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:45.477795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd250d-9295-4e9c-bf69-89dd3fb4b448 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:45.490668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd250d-9295-4e9c-bf69-89dd3fb4b448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-9cbd250d-9295-4e9c-bf69-89dd3fb4b448 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:45.495781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:45.496697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:45.497964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:45.502566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 116/241] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:45.504640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a865fa-ad58-4269-a11a-d2fe5dd4f059 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:45.504933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a865fa-ad58-4269-a11a-d2fe5dd4f059 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:45.505204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a865fa-ad58-4269-a11a-d2fe5dd4f059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:45.529931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a1a865fa-ad58-4269-a11a-d2fe5dd4f059 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:45.531290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a1a865fa-ad58-4269-a11a-d2fe5dd4f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:45.545601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:45.592992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:45.601203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571d2da-1a09-46b2-a688-54025d4ff783 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:45.626795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 117/242] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:45.628090 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a865fa-ad58-4269-a11a-d2fe5dd4f059 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:45.655276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358273-1db1-45c8-a828-318b04a826ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:45.655805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358273-1db1-45c8-a828-318b04a826ea None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:45.656082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358273-1db1-45c8-a828-318b04a826ea None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:45.656362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358273-1db1-45c8-a828-318b04a826ea None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:45.657077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97358273-1db1-45c8-a828-318b04a826ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:45.658691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97358273-1db1-45c8-a828-318b04a826ea None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:45.703678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 126/243] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:03:45 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:45.805972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a865fa-ad58-4269-a11a-d2fe5dd4f059 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:45.845341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:45.846380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:45.846885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:45.847352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:45.848231 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:45.848773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:45.850399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:45.860419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 118/244] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:45.868191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358273-1db1-45c8-a828-318b04a826ea None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:46.041190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358273-1db1-45c8-a828-318b04a826ea None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:46.058290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:46.084551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3360c8-dba3-4c23-808b-073a388d0c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:46.085214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3360c8-dba3-4c23-808b-073a388d0c36 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.085487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3360c8-dba3-4c23-808b-073a388d0c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.085752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3360c8-dba3-4c23-808b-073a388d0c36 None admin] PATH_INFO: `/v3/users/fd08d695035147fbb49f0b3364772c22` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.092789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:46.108556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3360c8-dba3-4c23-808b-073a388d0c36 None admin] RBAC: Authorizing `identity:get_user(user_id=fd08d695035147fbb49f0b3364772c22)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:46.109676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3360c8-dba3-4c23-808b-073a388d0c36 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:46.111666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 127/245] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:45 2022] GET /identity/v3/users/fd08d695035147fbb49f0b3364772c22 => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:46.117681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3360c8-dba3-4c23-808b-073a388d0c36 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:46.164606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:46.251952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde13fde-3584-4df7-ade4-791849112f9f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:46.268675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3360c8-dba3-4c23-808b-073a388d0c36 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:46.303805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2c764-2a36-4db7-8d24-4d18b982b7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:46.304841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c764-2a36-4db7-8d24-4d18b982b7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.305125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c764-2a36-4db7-8d24-4d18b982b7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.305669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c764-2a36-4db7-8d24-4d18b982b7c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.318596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da67d14-d1bb-469c-a07f-ffcf692faca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:46.319288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da67d14-d1bb-469c-a07f-ffcf692faca5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.320747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da67d14-d1bb-469c-a07f-ffcf692faca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.320747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da67d14-d1bb-469c-a07f-ffcf692faca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.323811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2c764-2a36-4db7-8d24-4d18b982b7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-75c2c764-2a36-4db7-8d24-4d18b982b7c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:46.332124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:46.333283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:46.334280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:46.337218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 119/246] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:46.339075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39ae8-d919-4e0d-8751-220b435e39f8 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.339769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39ae8-d919-4e0d-8751-220b435e39f8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.340246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be39ae8-d919-4e0d-8751-220b435e39f8 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.342071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 120/247] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:46.345421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39ae8-d919-4e0d-8751-220b435e39f8 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:46.379511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:46.420303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da67d14-d1bb-469c-a07f-ffcf692faca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:46.422606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da67d14-d1bb-469c-a07f-ffcf692faca5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:46.478973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:46.492595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be39ae8-d919-4e0d-8751-220b435e39f8 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:46.544205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892b0d3-58ac-4672-88f9-1f5e071526d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:46.545902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b0d3-58ac-4672-88f9-1f5e071526d5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.546676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b0d3-58ac-4672-88f9-1f5e071526d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.546958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b0d3-58ac-4672-88f9-1f5e071526d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.547939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892b0d3-58ac-4672-88f9-1f5e071526d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:46.548246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892b0d3-58ac-4672-88f9-1f5e071526d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:46.550892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892b0d3-58ac-4672-88f9-1f5e071526d5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:46.556493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 128/248] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:46.557855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31140-5052-4132-a307-24237b819a9d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.558140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31140-5052-4132-a307-24237b819a9d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.558410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f31140-5052-4132-a307-24237b819a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.563098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 121/249] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:46.564484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adbb88-a358-4f21-862f-b4874713c814 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.564797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adbb88-a358-4f21-862f-b4874713c814 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.565075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adbb88-a358-4f21-862f-b4874713c814 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.568668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 122/250] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:46.569503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adbb88-a358-4f21-862f-b4874713c814 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:46.595961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a7f31140-5052-4132-a307-24237b819a9d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:46.598055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a7f31140-5052-4132-a307-24237b819a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:46.710702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adbb88-a358-4f21-862f-b4874713c814 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:46.720992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 129/251] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:46.722542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f31140-5052-4132-a307-24237b819a9d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:46.726725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:46.772941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751153f-0bec-4f90-b4c4-fc01ccc0df7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:46.774006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751153f-0bec-4f90-b4c4-fc01ccc0df7b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.774497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751153f-0bec-4f90-b4c4-fc01ccc0df7b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.774982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751153f-0bec-4f90-b4c4-fc01ccc0df7b None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.777330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751153f-0bec-4f90-b4c4-fc01ccc0df7b None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:46.780135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751153f-0bec-4f90-b4c4-fc01ccc0df7b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:46.783640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 123/252] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:46 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:46.785748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ced5e-e2d7-4c14-8685-dd815bf51f98 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.786242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ced5e-e2d7-4c14-8685-dd815bf51f98 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.786713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ced5e-e2d7-4c14-8685-dd815bf51f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.816222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e43ced5e-e2d7-4c14-8685-dd815bf51f98 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:46.817358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e43ced5e-e2d7-4c14-8685-dd815bf51f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:46.867586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:46.899523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f31140-5052-4132-a307-24237b819a9d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:46.951305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 124/253] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:46.952185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ced5e-e2d7-4c14-8685-dd815bf51f98 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:46.963405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d886-5f12-439b-b001-5eca7106848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:46.964043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d886-5f12-439b-b001-5eca7106848e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:46.964317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d886-5f12-439b-b001-5eca7106848e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:46.964597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d886-5f12-439b-b001-5eca7106848e None admin] PATH_INFO: `/v3/users/24f7a30de104416e9a1f9c3a32e12b63` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:46.968035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494d886-5f12-439b-b001-5eca7106848e None admin] RBAC: Authorizing `identity:get_user(user_id=24f7a30de104416e9a1f9c3a32e12b63)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:46.969506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494d886-5f12-439b-b001-5eca7106848e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:46.971277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 130/254] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:46 2022] GET /identity/v3/users/24f7a30de104416e9a1f9c3a32e12b63 => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:46.979744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d886-5f12-439b-b001-5eca7106848e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:47.185101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:47.236038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ced5e-e2d7-4c14-8685-dd815bf51f98 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:47.266164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d886-5f12-439b-b001-5eca7106848e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:47.313389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:47.314164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:47.314286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:47.314642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:47.315516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] RBAC: Adding query filter params (role.id=424ecb0deee84a068044178a8ee9ad52, scope.project.id=362cde2d45474fbcadbd8b85adbbac97, user.id=fd08d695035147fbb49f0b3364772c22) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:47.315819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:47.324596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:47.324596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 125/255] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:46 2022] GET /identity/v3/role_assignments?user.id=fd08d695035147fbb49f0b3364772c22&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=424ecb0deee84a068044178a8ee9ad52 => generated 259 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:47.330036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:47.359707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c25be0-cda4-4a90-ad47-6465f12b637d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:47.361282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c25be0-cda4-4a90-ad47-6465f12b637d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:47.362015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c25be0-cda4-4a90-ad47-6465f12b637d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:47.362750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c25be0-cda4-4a90-ad47-6465f12b637d None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:47.372547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c25be0-cda4-4a90-ad47-6465f12b637d None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:47.375288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c25be0-cda4-4a90-ad47-6465f12b637d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:47.378797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 131/256] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:46 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:47.379957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c25be0-cda4-4a90-ad47-6465f12b637d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:47.380193 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:47.599095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c25be0-cda4-4a90-ad47-6465f12b637d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:47.635684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef7924-f06f-4ed7-9bd8-f61b1a9f7e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:47.636344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef7924-f06f-4ed7-9bd8-f61b1a9f7e59 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:47.636668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef7924-f06f-4ed7-9bd8-f61b1a9f7e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:47.636943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef7924-f06f-4ed7-9bd8-f61b1a9f7e59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:47.647689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef7924-f06f-4ed7-9bd8-f61b1a9f7e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:47.650545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caf40a-b978-4b78-a38b-baf05f6789ee None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5fef7924-f06f-4ed7-9bd8-f61b1a9f7e59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:47.652625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:47.654040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:03:47.655143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:47.661030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 132/257] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:47.661030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:47.670203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef7924-f06f-4ed7-9bd8-f61b1a9f7e59 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:47.713519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bb0ab-d45a-465d-8f42-95dd68f7e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:47.715034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bb0ab-d45a-465d-8f42-95dd68f7e876 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:47.715872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bb0ab-d45a-465d-8f42-95dd68f7e876 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:47.716652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bb0ab-d45a-465d-8f42-95dd68f7e876 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:47.760604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bb0ab-d45a-465d-8f42-95dd68f7e876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-2d9bb0ab-d45a-465d-8f42-95dd68f7e876 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:47.765414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:47.766329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:03:47.767175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:47.768691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 126/258] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:03:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:47.773842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bb0ab-d45a-465d-8f42-95dd68f7e876 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:47.813783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef7924-f06f-4ed7-9bd8-f61b1a9f7e59 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:47.852128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ceacb-e196-4cc1-96f6-8c35046d8d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:47.852883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ceacb-e196-4cc1-96f6-8c35046d8d38 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:47.853159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ceacb-e196-4cc1-96f6-8c35046d8d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:47.853509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ceacb-e196-4cc1-96f6-8c35046d8d38 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97/users/24f7a30de104416e9a1f9c3a32e12b63/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:47.855398 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ceacb-e196-4cc1-96f6-8c35046d8d38 None admin] RBAC: Authorizing `identity:create_grant(project_id=362cde2d45474fbcadbd8b85adbbac97, user_id=24f7a30de104416e9a1f9c3a32e12b63, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:47.857826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ceacb-e196-4cc1-96f6-8c35046d8d38 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:47.872402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 133/259] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:47 2022] PUT /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97/users/24f7a30de104416e9a1f9c3a32e12b63/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:47.873279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ceacb-e196-4cc1-96f6-8c35046d8d38 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:47.940619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:47.990618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ceacb-e196-4cc1-96f6-8c35046d8d38 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:48.031461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:48.032090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:48.032366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:48.032660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:48.033339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:48.033637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:48.035281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:48.045414 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 134/260] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:03:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:48.068798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:48.089913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:48.172532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bb0ab-d45a-465d-8f42-95dd68f7e876 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:48.192119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d711d-bbb1-4411-bd97-80c26f251112 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:48.215180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:48.235283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171e513-1c0c-4f5f-9727-6a0f669eb1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:48.235283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171e513-1c0c-4f5f-9727-6a0f669eb1bd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:48.235283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171e513-1c0c-4f5f-9727-6a0f669eb1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:48.236144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171e513-1c0c-4f5f-9727-6a0f669eb1bd None admin] PATH_INFO: `/v3/users/7b5dedda01774f71929a65c06a137247` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:48.238455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171e513-1c0c-4f5f-9727-6a0f669eb1bd None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dedda01774f71929a65c06a137247)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:48.239940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171e513-1c0c-4f5f-9727-6a0f669eb1bd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:48.245819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 135/261] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:48 2022] GET /identity/v3/users/7b5dedda01774f71929a65c06a137247 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:48.251804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171e513-1c0c-4f5f-9727-6a0f669eb1bd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:48.282821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:48.283451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:48.285423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:48.285712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:48.286402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:48.286702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:48.290883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:48.309617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 127/262] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 541 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:48.316230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:48.436925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31ff86-696f-4577-b6d4-307a961102fd None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:48.460933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171e513-1c0c-4f5f-9727-6a0f669eb1bd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:48.476175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf48fe-6d52-4373-acac-cecf1af78318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:48.476841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf48fe-6d52-4373-acac-cecf1af78318 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:48.477121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf48fe-6d52-4373-acac-cecf1af78318 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:48.477390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf48fe-6d52-4373-acac-cecf1af78318 None admin] PATH_INFO: `/v3/users/7b5dedda01774f71929a65c06a137247` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:48.478577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf48fe-6d52-4373-acac-cecf1af78318 None admin] RBAC: Authorizing `identity:get_user(user_id=7b5dedda01774f71929a65c06a137247)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:48.480047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf48fe-6d52-4373-acac-cecf1af78318 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:48.482597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 128/263] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:48 2022] GET /identity/v3/users/7b5dedda01774f71929a65c06a137247 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:48.488913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf48fe-6d52-4373-acac-cecf1af78318 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:48.506892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:48.591429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817e374-7403-4fc0-a364-66fd00b667dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:48.592476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817e374-7403-4fc0-a364-66fd00b667dc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:48.593002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817e374-7403-4fc0-a364-66fd00b667dc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:48.593462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817e374-7403-4fc0-a364-66fd00b667dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:48.605331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf48fe-6d52-4373-acac-cecf1af78318 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:48.645079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49074a89-c21b-488b-a97e-19f3506879a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:48.646099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074a89-c21b-488b-a97e-19f3506879a1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:48.646476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074a89-c21b-488b-a97e-19f3506879a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:48.646844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074a89-c21b-488b-a97e-19f3506879a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:48.726060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49074a89-c21b-488b-a97e-19f3506879a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:48.727815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49074a89-c21b-488b-a97e-19f3506879a1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:48.763864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:48.789062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:48.834191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817e374-7403-4fc0-a364-66fd00b667dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:48.836960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817e374-7403-4fc0-a364-66fd00b667dc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:49.096331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 129/264] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:49.098399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d962e-f745-4e6f-87f4-c34d7c3f7552 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:49.098883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d962e-f745-4e6f-87f4-c34d7c3f7552 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:49.099339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d962e-f745-4e6f-87f4-c34d7c3f7552 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:49.105937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 130/265] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:49.107379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d962e-f745-4e6f-87f4-c34d7c3f7552 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:49.109255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 136/266] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 858 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:03:49.113391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4824-ce9f-42f8-8380-d5a801dcf437 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:49.116133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4824-ce9f-42f8-8380-d5a801dcf437 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:49.119828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4824-ce9f-42f8-8380-d5a801dcf437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:49.154793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-54df4824-ce9f-42f8-8380-d5a801dcf437 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:49.157312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-54df4824-ce9f-42f8-8380-d5a801dcf437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:49.286733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 137/267] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:49.288871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df4824-ce9f-42f8-8380-d5a801dcf437 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:49.333413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:49.353356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d962e-f745-4e6f-87f4-c34d7c3f7552 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:49.382703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:49.465333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a3af4-d76c-45e5-b5d8-46843fed5679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:49.466573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a3af4-d76c-45e5-b5d8-46843fed5679 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:49.466957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a3af4-d76c-45e5-b5d8-46843fed5679 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:49.467104 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a3af4-d76c-45e5-b5d8-46843fed5679 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:49.468416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a3af4-d76c-45e5-b5d8-46843fed5679 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:49.469963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a3af4-d76c-45e5-b5d8-46843fed5679 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:49.476430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 131/268] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:49 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:49.477769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a3af4-d76c-45e5-b5d8-46843fed5679 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:49.494302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df4824-ce9f-42f8-8380-d5a801dcf437 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:49.563310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45024a-8968-47b9-bc17-7e17164f729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:49.564863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45024a-8968-47b9-bc17-7e17164f729d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:49.565611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45024a-8968-47b9-bc17-7e17164f729d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:49.566409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45024a-8968-47b9-bc17-7e17164f729d None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:49.568536 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45024a-8968-47b9-bc17-7e17164f729d None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:49.571614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45024a-8968-47b9-bc17-7e17164f729d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:49.576044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 138/269] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:49 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:49.600533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45024a-8968-47b9-bc17-7e17164f729d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:49.707773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a3af4-d76c-45e5-b5d8-46843fed5679 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:49.776331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fdc2b-6ce6-4e62-8800-bb0747c032e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:49.776979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fdc2b-6ce6-4e62-8800-bb0747c032e4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:49.777251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fdc2b-6ce6-4e62-8800-bb0747c032e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:49.777591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fdc2b-6ce6-4e62-8800-bb0747c032e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:49.779268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fdc2b-6ce6-4e62-8800-bb0747c032e4 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=362cde2d45474fbcadbd8b85adbbac97, user.id=7b5dedda01774f71929a65c06a137247) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:49.779268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fdc2b-6ce6-4e62-8800-bb0747c032e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:49.781261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fdc2b-6ce6-4e62-8800-bb0747c032e4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:49.787583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 132/270] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:49 2022] GET /identity/v3/role_assignments?user.id=7b5dedda01774f71929a65c06a137247&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:49.789044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:49.795045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf5fbb-df52-4541-8b0f-e0d7aa187455 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:49.795482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf5fbb-df52-4541-8b0f-e0d7aa187455 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:49.795619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf5fbb-df52-4541-8b0f-e0d7aa187455 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:49.797223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 133/271] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:49.805366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97db91-c736-432c-a0bd-b5ef2ada7b00 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:49.805764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97db91-c736-432c-a0bd-b5ef2ada7b00 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:49.806040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97db91-c736-432c-a0bd-b5ef2ada7b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:49.829100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45024a-8968-47b9-bc17-7e17164f729d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:49.842688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6f97db91-c736-432c-a0bd-b5ef2ada7b00 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:49.843946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6f97db91-c736-432c-a0bd-b5ef2ada7b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:49.890464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:49.914448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:49.915147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:49.915403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:49.915675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:49.916412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] RBAC: Adding query filter params (role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=362cde2d45474fbcadbd8b85adbbac97, user.id=7b5dedda01774f71929a65c06a137247) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:49.916753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:49.918282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:49.922503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 134/272] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:49.923559 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97db91-c736-432c-a0bd-b5ef2ada7b00 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:49.934157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 139/273] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:49 2022] GET /identity/v3/role_assignments?user.id=7b5dedda01774f71929a65c06a137247&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=79dbf286cf884619af62f33ea75c52bf => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:50.042977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97db91-c736-432c-a0bd-b5ef2ada7b00 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:50.086359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b605098-3dd1-47cf-ba25-ba34907d0aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:50.087004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b605098-3dd1-47cf-ba25-ba34907d0aae None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:50.087279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b605098-3dd1-47cf-ba25-ba34907d0aae None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:50.087546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b605098-3dd1-47cf-ba25-ba34907d0aae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:50.088694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b605098-3dd1-47cf-ba25-ba34907d0aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:50.090201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b605098-3dd1-47cf-ba25-ba34907d0aae None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:50.093069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 135/274] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:50.098341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:50.247583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811aa7-6541-4e21-a153-be1cd939c772 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:50.290363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e004140-ac8d-4e78-8e1b-9680215514dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:50.291277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004140-ac8d-4e78-8e1b-9680215514dd None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:50.291728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004140-ac8d-4e78-8e1b-9680215514dd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:50.292169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004140-ac8d-4e78-8e1b-9680215514dd None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:50.295380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e004140-ac8d-4e78-8e1b-9680215514dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:50.297191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e004140-ac8d-4e78-8e1b-9680215514dd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:50.323523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b605098-3dd1-47cf-ba25-ba34907d0aae None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:50.343992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 140/275] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:03:50 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:50.391089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:50.469861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b605098-3dd1-47cf-ba25-ba34907d0aae None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:50.511880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec33ac-1983-4d26-8a38-984cab80238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:50.512521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec33ac-1983-4d26-8a38-984cab80238f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:50.512835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec33ac-1983-4d26-8a38-984cab80238f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:50.513158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec33ac-1983-4d26-8a38-984cab80238f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:50.524623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec33ac-1983-4d26-8a38-984cab80238f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-c5ec33ac-1983-4d26-8a38-984cab80238f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:50.529758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:50.530726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:50.531529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 136/276] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:03:50.536736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e004140-ac8d-4e78-8e1b-9680215514dd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:50.558137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:50.668268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e004140-ac8d-4e78-8e1b-9680215514dd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:50.694338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23407a5-54b1-4530-b033-d29d488e2ea8 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:50.694836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23407a5-54b1-4530-b033-d29d488e2ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:50.695295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23407a5-54b1-4530-b033-d29d488e2ea8 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:50.697374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 137/277] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:50.706438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d11e-d1cd-4b2a-9816-ff6c5d9db2a9 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:50.707038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d11e-d1cd-4b2a-9816-ff6c5d9db2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:50.707701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124d11e-d1cd-4b2a-9816-ff6c5d9db2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:50.709258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:50.710249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:50.710786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:50.711229 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:50.712078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:50.715156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:50.715156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:50.725550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 141/278] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:50.731199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:50.739286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-2124d11e-d1cd-4b2a-9816-ff6c5d9db2a9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:50.740919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2124d11e-d1cd-4b2a-9816-ff6c5d9db2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:50.829277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 138/279] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:50.840594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:50.884735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76faf6f-5a07-46b3-92a1-b44e8dbe3c11 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:50.929280 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8dc93-6242-48aa-8eb7-598fa51db967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:50.929913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8dc93-6242-48aa-8eb7-598fa51db967 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:50.930186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8dc93-6242-48aa-8eb7-598fa51db967 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:50.930463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8dc93-6242-48aa-8eb7-598fa51db967 None admin] PATH_INFO: `/v3/users/7197913dbfe04fddb955a30a5a90696e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:50.951776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8dc93-6242-48aa-8eb7-598fa51db967 None admin] RBAC: Authorizing `identity:get_user(user_id=7197913dbfe04fddb955a30a5a90696e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:50.953285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8dc93-6242-48aa-8eb7-598fa51db967 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:50.959281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 142/280] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:50 2022] GET /identity/v3/users/7197913dbfe04fddb955a30a5a90696e => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:50.965001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d11e-d1cd-4b2a-9816-ff6c5d9db2a9 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:51.020237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:51.130794 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124d11e-d1cd-4b2a-9816-ff6c5d9db2a9 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:51.187723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfd681-5e82-49b8-88e3-996483a2ac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:51.188895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfd681-5e82-49b8-88e3-996483a2ac98 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:51.189381 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfd681-5e82-49b8-88e3-996483a2ac98 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:51.189891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfd681-5e82-49b8-88e3-996483a2ac98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:51.190825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfd681-5e82-49b8-88e3-996483a2ac98 None admin] RBAC: Adding query filter params (role.id=c2bcd89a362f4fb3a73e0f8ada320802, user.id=7197913dbfe04fddb955a30a5a90696e, scope.system=all) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:51.191332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfd681-5e82-49b8-88e3-996483a2ac98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:51.193049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfd681-5e82-49b8-88e3-996483a2ac98 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:51.202396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 139/281] 10.222.0.95 () {56 vars in 1308 bytes} [Wed Aug 31 19:03:50 2022] GET /identity/v3/role_assignments?user.id=7197913dbfe04fddb955a30a5a90696e&scope.system=all&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:51.230010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8dc93-6242-48aa-8eb7-598fa51db967 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:51.265565 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e5322-f5ac-4e30-8202-8c1df8cd200c None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:51.266632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e5322-f5ac-4e30-8202-8c1df8cd200c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:51.267112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e5322-f5ac-4e30-8202-8c1df8cd200c None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:51.269772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 140/282] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:51.276301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ead7-e2f5-475b-ad36-e06c959434db None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:51.276948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ead7-e2f5-475b-ad36-e06c959434db None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:51.277434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ead7-e2f5-475b-ad36-e06c959434db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:51.311449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e175ead7-e2f5-475b-ad36-e06c959434db None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:51.312891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e175ead7-e2f5-475b-ad36-e06c959434db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:51.382585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8dc93-6242-48aa-8eb7-598fa51db967 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:51.413810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:51.426791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d1b79-66fc-486f-8734-952eea166c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:51.427730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 141/283] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:51.428844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1b79-66fc-486f-8734-952eea166c17 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:51.429428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1b79-66fc-486f-8734-952eea166c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:51.429915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d1b79-66fc-486f-8734-952eea166c17 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:51.442670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d1b79-66fc-486f-8734-952eea166c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8b9d1b79-66fc-486f-8734-952eea166c17 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:51.448115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:51.449087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:51.450786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:51.458157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 143/284] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:51.459995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175ead7-e2f5-475b-ad36-e06c959434db None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:51.509146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:51.612869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175ead7-e2f5-475b-ad36-e06c959434db None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:51.657361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:51.657898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:51.658177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:51.658451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:51.659225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:51.659530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:51.660999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:51.672046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 142/285] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:51.678103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d1b79-66fc-486f-8734-952eea166c17 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:51.815989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:51.855065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d1b79-66fc-486f-8734-952eea166c17 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:51.908292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ccbda-fed0-4f05-87ca-df0dd15a6c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:51.910671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ccbda-fed0-4f05-87ca-df0dd15a6c62 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:51.910953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ccbda-fed0-4f05-87ca-df0dd15a6c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:51.911228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ccbda-fed0-4f05-87ca-df0dd15a6c62 None admin] PATH_INFO: `/v3/users/fd08d695035147fbb49f0b3364772c22` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:51.913245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ccbda-fed0-4f05-87ca-df0dd15a6c62 None admin] RBAC: Authorizing `identity:get_user(user_id=fd08d695035147fbb49f0b3364772c22)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:51.914694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ccbda-fed0-4f05-87ca-df0dd15a6c62 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:51.916282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 144/286] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:51 2022] GET /identity/v3/users/fd08d695035147fbb49f0b3364772c22 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:51.923785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ccbda-fed0-4f05-87ca-df0dd15a6c62 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:51.976961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:52.000430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614a622-ac8d-4046-994e-c0605d7da3ac None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:52.034414 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:52.084312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43024fbe-409e-4f61-adca-6d025316b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:52.085436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024fbe-409e-4f61-adca-6d025316b260 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:52.085924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024fbe-409e-4f61-adca-6d025316b260 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:52.086491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024fbe-409e-4f61-adca-6d025316b260 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:52.092146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ccbda-fed0-4f05-87ca-df0dd15a6c62 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:52.116271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43024fbe-409e-4f61-adca-6d025316b260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:52.132591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcb8b0-3aa6-4904-b289-3d4741230732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:52.132591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb8b0-3aa6-4904-b289-3d4741230732 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:52.132591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb8b0-3aa6-4904-b289-3d4741230732 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:52.132591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcb8b0-3aa6-4904-b289-3d4741230732 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:52.133143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcb8b0-3aa6-4904-b289-3d4741230732 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:52.133143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcb8b0-3aa6-4904-b289-3d4741230732 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:52.133143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 145/287] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:51 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-43024fbe-409e-4f61-adca-6d025316b260 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:52.137463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:52.138516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:52.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:52.149760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 143/288] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:52.152786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43024fbe-409e-4f61-adca-6d025316b260 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:52.152901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcb8b0-3aa6-4904-b289-3d4741230732 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:52.259136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcb8b0-3aa6-4904-b289-3d4741230732 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:52.289417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:52.297774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:52.298389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:52.298664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:52.298929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:52.299593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:52.299892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:52.304519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:52.314503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 146/289] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:52.319928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:52.399471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43024fbe-409e-4f61-adca-6d025316b260 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:52.426182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2f2c0-c26f-4b48-8b12-7df3ef4fed90 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:52.461345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b56ec8-d707-432c-840d-0423487c9d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:52.462017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b56ec8-d707-432c-840d-0423487c9d2b None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:52.462283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b56ec8-d707-432c-840d-0423487c9d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:52.462563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b56ec8-d707-432c-840d-0423487c9d2b None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97/users/fd08d695035147fbb49f0b3364772c22/roles/424ecb0deee84a068044178a8ee9ad52` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:52.464902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b56ec8-d707-432c-840d-0423487c9d2b None admin] RBAC: Authorizing `identity:create_grant(project_id=362cde2d45474fbcadbd8b85adbbac97, user_id=fd08d695035147fbb49f0b3364772c22, role_id=424ecb0deee84a068044178a8ee9ad52)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:52.466997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b56ec8-d707-432c-840d-0423487c9d2b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:52.488074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 144/290] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:03:52 2022] PUT /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97/users/fd08d695035147fbb49f0b3364772c22/roles/424ecb0deee84a068044178a8ee9ad52 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:52.503701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba952f-1ae9-4491-8dee-dc7197771850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:52.504420 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba952f-1ae9-4491-8dee-dc7197771850 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:52.504771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba952f-1ae9-4491-8dee-dc7197771850 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:52.505087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba952f-1ae9-4491-8dee-dc7197771850 None admin] PATH_INFO: `/v3/users/24f7a30de104416e9a1f9c3a32e12b63` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:52.506386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba952f-1ae9-4491-8dee-dc7197771850 None admin] RBAC: Authorizing `identity:get_user(user_id=24f7a30de104416e9a1f9c3a32e12b63)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:52.507933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba952f-1ae9-4491-8dee-dc7197771850 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:52.509750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 147/291] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:52 2022] GET /identity/v3/users/24f7a30de104416e9a1f9c3a32e12b63 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:52.516420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b56ec8-d707-432c-840d-0423487c9d2b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:52.528954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:52.641023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b56ec8-d707-432c-840d-0423487c9d2b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:52.679660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c4904-85d0-455d-bc42-48fdfab000f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:52.680348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c4904-85d0-455d-bc42-48fdfab000f1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:52.680656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c4904-85d0-455d-bc42-48fdfab000f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:52.680929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c4904-85d0-455d-bc42-48fdfab000f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:52.759388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c4904-85d0-455d-bc42-48fdfab000f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:52.761434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c4904-85d0-455d-bc42-48fdfab000f1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:52.774946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:52.875449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 145/292] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:03:52.881679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba952f-1ae9-4491-8dee-dc7197771850 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:52.971940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:53.052901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba952f-1ae9-4491-8dee-dc7197771850 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:53.066133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a55239-c7e0-4427-a123-976512ab2e68 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.066461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a55239-c7e0-4427-a123-976512ab2e68 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.066683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a55239-c7e0-4427-a123-976512ab2e68 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.068170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 146/293] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:53.074706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8034-5d24-44a1-b631-084c6e035b53 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.075600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8034-5d24-44a1-b631-084c6e035b53 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.076113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8034-5d24-44a1-b631-084c6e035b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.112124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-889c8034-5d24-44a1-b631-084c6e035b53 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:53.114154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-889c8034-5d24-44a1-b631-084c6e035b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:53.168041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d9e00-6518-4f4f-b40c-c61163875de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:53.169696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d9e00-6518-4f4f-b40c-c61163875de3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.170171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d9e00-6518-4f4f-b40c-c61163875de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.170666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d9e00-6518-4f4f-b40c-c61163875de3 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.174376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d9e00-6518-4f4f-b40c-c61163875de3 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:53.177402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d9e00-6518-4f4f-b40c-c61163875de3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:53.191268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 148/294] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:52 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:53.192905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d9e00-6518-4f4f-b40c-c61163875de3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:53.299886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 147/295] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:53.370137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d9e00-6518-4f4f-b40c-c61163875de3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:53.404754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:53.430978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5686ca-1791-45d8-a0de-1813db5195fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:53.431852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5686ca-1791-45d8-a0de-1813db5195fa None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.432356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5686ca-1791-45d8-a0de-1813db5195fa None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.432794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5686ca-1791-45d8-a0de-1813db5195fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.433671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5686ca-1791-45d8-a0de-1813db5195fa None admin] RBAC: Adding query filter params (role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=362cde2d45474fbcadbd8b85adbbac97, user.id=24f7a30de104416e9a1f9c3a32e12b63) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:53.434125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5686ca-1791-45d8-a0de-1813db5195fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:53.436604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5686ca-1791-45d8-a0de-1813db5195fa None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:53.443674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 149/296] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:53 2022] GET /identity/v3/role_assignments?user.id=24f7a30de104416e9a1f9c3a32e12b63&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:53.614931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3833e54-f81a-4901-b736-7b4a22e03e8a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.615876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3833e54-f81a-4901-b736-7b4a22e03e8a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.616305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3833e54-f81a-4901-b736-7b4a22e03e8a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.618177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 148/297] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:53.632739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3725a-a009-4f98-b634-254ba8987b58 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.633239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3725a-a009-4f98-b634-254ba8987b58 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.633673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3725a-a009-4f98-b634-254ba8987b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.661263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-37e3725a-a009-4f98-b634-254ba8987b58 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:53.662757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-37e3725a-a009-4f98-b634-254ba8987b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:53.666123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3833e54-f81a-4901-b736-7b4a22e03e8a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:53.719367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:53.790230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3833e54-f81a-4901-b736-7b4a22e03e8a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:53.797045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 150/298] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:53.830985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f2b37-c4e8-4444-90bd-f6c2bd233faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:53.831611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f2b37-c4e8-4444-90bd-f6c2bd233faa None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.831886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f2b37-c4e8-4444-90bd-f6c2bd233faa None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.832150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f2b37-c4e8-4444-90bd-f6c2bd233faa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.833305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f2b37-c4e8-4444-90bd-f6c2bd233faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:53.834762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f2b37-c4e8-4444-90bd-f6c2bd233faa None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:53.836333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 149/299] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:53.841893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3725a-a009-4f98-b634-254ba8987b58 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:53.879705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb15e2-71cd-4b60-a7c6-10cf5c061f56 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.880265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb15e2-71cd-4b60-a7c6-10cf5c061f56 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.880731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb15e2-71cd-4b60-a7c6-10cf5c061f56 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.882513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 150/300] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:53.902129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162f501-efd5-461e-bec7-efd6ce34a96a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.902707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162f501-efd5-461e-bec7-efd6ce34a96a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:53.903201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162f501-efd5-461e-bec7-efd6ce34a96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:53.928061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d162f501-efd5-461e-bec7-efd6ce34a96a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:53.929663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d162f501-efd5-461e-bec7-efd6ce34a96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:53.954608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3725a-a009-4f98-b634-254ba8987b58 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:53.970009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:53.996677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43240e93-c78e-4d7d-bbb1-a8c612b349c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:53.998257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240e93-c78e-4d7d-bbb1-a8c612b349c8 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:53.999538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240e93-c78e-4d7d-bbb1-a8c612b349c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:54.000189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43240e93-c78e-4d7d-bbb1-a8c612b349c8 None admin] PATH_INFO: `/v3/groups` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:54.002300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43240e93-c78e-4d7d-bbb1-a8c612b349c8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:54.005161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43240e93-c78e-4d7d-bbb1-a8c612b349c8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:54.023905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 151/301] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:54.026978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 151/302] 10.222.0.95 () {60 vars in 1134 bytes} [Wed Aug 31 19:03:53 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:54.126447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162f501-efd5-461e-bec7-efd6ce34a96a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:54.180021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:54.283953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162f501-efd5-461e-bec7-efd6ce34a96a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:54.313175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43240e93-c78e-4d7d-bbb1-a8c612b349c8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:54.348407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43383062-6d17-4c00-af2a-f1c4e8ab5381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:54.349880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43383062-6d17-4c00-af2a-f1c4e8ab5381 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:54.350701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43383062-6d17-4c00-af2a-f1c4e8ab5381 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:54.351376 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43383062-6d17-4c00-af2a-f1c4e8ab5381 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:54.372735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43383062-6d17-4c00-af2a-f1c4e8ab5381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-43383062-6d17-4c00-af2a-f1c4e8ab5381 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:54.383942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:54.385895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:54.387694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:54.392039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 152/303] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:54.401530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43383062-6d17-4c00-af2a-f1c4e8ab5381 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:54.440981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43240e93-c78e-4d7d-bbb1-a8c612b349c8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:54.483770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f78fad-42b6-4e3c-9e09-28117e46a3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:54.484608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f78fad-42b6-4e3c-9e09-28117e46a3ce None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:54.485015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f78fad-42b6-4e3c-9e09-28117e46a3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:54.485373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f78fad-42b6-4e3c-9e09-28117e46a3ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:54.500302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f78fad-42b6-4e3c-9e09-28117e46a3ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-69f78fad-42b6-4e3c-9e09-28117e46a3ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:54.506790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:54.508720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:54.510485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:54.514770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 152/304] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:03:54.516557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f78fad-42b6-4e3c-9e09-28117e46a3ce None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:54.573234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:54.575806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:54.581952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43383062-6d17-4c00-af2a-f1c4e8ab5381 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:54.626110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:54.627130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:54.627130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:54.627361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:54.627965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:54.628271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:54.629716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:54.646449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 153/305] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:54.650338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:54.684389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f78fad-42b6-4e3c-9e09-28117e46a3ce None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:54.727741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:54.728817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:54.729325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:54.729788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:54.730767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:54.731936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:54.733870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:54.747582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 153/306] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:54.753815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:54.811836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1c71e-0cb8-4fd8-a3de-a8ed8f4ee844 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:54.857689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540bb9-a1fb-4195-892d-ddfb925b3de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:54.858650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540bb9-a1fb-4195-892d-ddfb925b3de4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:54.859108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540bb9-a1fb-4195-892d-ddfb925b3de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:54.859384 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540bb9-a1fb-4195-892d-ddfb925b3de4 None admin] PATH_INFO: `/v3/users/b931b49238ee4267bf80a18fb274e197` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:54.874294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:54.877330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540bb9-a1fb-4195-892d-ddfb925b3de4 None admin] RBAC: Authorizing `identity:get_user(user_id=b931b49238ee4267bf80a18fb274e197)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:54.879258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540bb9-a1fb-4195-892d-ddfb925b3de4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:54.881917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 154/307] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:54 2022] GET /identity/v3/users/b931b49238ee4267bf80a18fb274e197 => generated 293 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:54.883052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eebec-c888-4366-90cc-5922ac757714 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:54.888911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540bb9-a1fb-4195-892d-ddfb925b3de4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:54.918487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:54.940421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1dd64-670b-4feb-8eca-0e1cda3d9ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:54.941073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1dd64-670b-4feb-8eca-0e1cda3d9ccf None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:54.941355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1dd64-670b-4feb-8eca-0e1cda3d9ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:54.941628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1dd64-670b-4feb-8eca-0e1cda3d9ccf None admin] PATH_INFO: `/v3/users/7197913dbfe04fddb955a30a5a90696e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:54.942796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1dd64-670b-4feb-8eca-0e1cda3d9ccf None admin] RBAC: Authorizing `identity:get_user(user_id=7197913dbfe04fddb955a30a5a90696e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:54.944263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1dd64-670b-4feb-8eca-0e1cda3d9ccf None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:54.946291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 154/308] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:54 2022] GET /identity/v3/users/7197913dbfe04fddb955a30a5a90696e => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:54.951854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1dd64-670b-4feb-8eca-0e1cda3d9ccf None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:55.033216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540bb9-a1fb-4195-892d-ddfb925b3de4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:55.068210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1dd64-670b-4feb-8eca-0e1cda3d9ccf None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:55.079129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaed90-02cf-4e3d-b3c0-d86b56ff5856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:55.080336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaed90-02cf-4e3d-b3c0-d86b56ff5856 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:55.080945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaed90-02cf-4e3d-b3c0-d86b56ff5856 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:55.081461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaed90-02cf-4e3d-b3c0-d86b56ff5856 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:55.082344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaed90-02cf-4e3d-b3c0-d86b56ff5856 None admin] RBAC: Adding query filter params (role.id=424ecb0deee84a068044178a8ee9ad52, user.id=b931b49238ee4267bf80a18fb274e197, scope.system=all) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:55.083030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaed90-02cf-4e3d-b3c0-d86b56ff5856 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:55.085425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaed90-02cf-4e3d-b3c0-d86b56ff5856 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:55.100813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 155/309] 10.222.0.95 () {56 vars in 1308 bytes} [Wed Aug 31 19:03:54 2022] GET /identity/v3/role_assignments?user.id=b931b49238ee4267bf80a18fb274e197&scope.system=all&role.id=424ecb0deee84a068044178a8ee9ad52 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:55.116446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f653d2-ee06-41c1-9c16-b95d03c475e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:55.117263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f653d2-ee06-41c1-9c16-b95d03c475e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:55.117613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f653d2-ee06-41c1-9c16-b95d03c475e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:55.117891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f653d2-ee06-41c1-9c16-b95d03c475e7 None admin] PATH_INFO: `/v3/system/users/7197913dbfe04fddb955a30a5a90696e/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:55.119552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f653d2-ee06-41c1-9c16-b95d03c475e7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7197913dbfe04fddb955a30a5a90696e, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:55.121036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f653d2-ee06-41c1-9c16-b95d03c475e7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:55.133952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 155/310] 10.222.0.95 () {58 vars in 1334 bytes} [Wed Aug 31 19:03:54 2022] PUT /identity/v3/system/users/7197913dbfe04fddb955a30a5a90696e/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:55.142914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa289-165a-45c8-b535-ec469df11042 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:55.143218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa289-165a-45c8-b535-ec469df11042 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:55.143466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa289-165a-45c8-b535-ec469df11042 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:55.145264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 156/311] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:55.152355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baa86-213f-48cc-b2fa-aa9ea35619f4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:55.152820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baa86-213f-48cc-b2fa-aa9ea35619f4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:55.153167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0baa86-213f-48cc-b2fa-aa9ea35619f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:55.174877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-bb0baa86-213f-48cc-b2fa-aa9ea35619f4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:55.176401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bb0baa86-213f-48cc-b2fa-aa9ea35619f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:55.219391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:55.288727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 156/312] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:55.567784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa289-165a-45c8-b535-ec469df11042 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:55.664835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:55.688303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa289-165a-45c8-b535-ec469df11042 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:55.732085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a81577-3415-4f03-a94a-9188abb03674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:55.733646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a81577-3415-4f03-a94a-9188abb03674 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:55.734357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a81577-3415-4f03-a94a-9188abb03674 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:55.734881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a81577-3415-4f03-a94a-9188abb03674 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:55.749681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a81577-3415-4f03-a94a-9188abb03674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-37a81577-3415-4f03-a94a-9188abb03674 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:55.754786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:55.755633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:55.756425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:55.758975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 157/313] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:55.765047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0baa86-213f-48cc-b2fa-aa9ea35619f4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:55.882099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0baa86-213f-48cc-b2fa-aa9ea35619f4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:55.918177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:55.919836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:55.920414 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:55.920921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:55.921774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:55.922336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:55.923889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:55.933585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 157/314] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:55.939597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a81577-3415-4f03-a94a-9188abb03674 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:56.070917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a81577-3415-4f03-a94a-9188abb03674 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:56.108753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f5e7-3f5e-4f12-8483-6c446a6f1522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:56.109772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f5e7-3f5e-4f12-8483-6c446a6f1522 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:56.110243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f5e7-3f5e-4f12-8483-6c446a6f1522 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:56.110703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f5e7-3f5e-4f12-8483-6c446a6f1522 None admin] PATH_INFO: `/v3/users/fd08d695035147fbb49f0b3364772c22` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:56.112092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208f5e7-3f5e-4f12-8483-6c446a6f1522 None admin] RBAC: Authorizing `identity:get_user(user_id=fd08d695035147fbb49f0b3364772c22)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:56.113821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208f5e7-3f5e-4f12-8483-6c446a6f1522 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:56.119670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 158/315] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:55 2022] GET /identity/v3/users/fd08d695035147fbb49f0b3364772c22 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:56.123605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:56.155090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:56.242455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134fbc6-737b-44a6-8b21-975204a1b139 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:56.279571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172fef4-eea3-4846-9dad-446897444f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:56.280678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172fef4-eea3-4846-9dad-446897444f3c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:56.281156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172fef4-eea3-4846-9dad-446897444f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:56.281617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172fef4-eea3-4846-9dad-446897444f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:56.359731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172fef4-eea3-4846-9dad-446897444f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:56.361571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172fef4-eea3-4846-9dad-446897444f3c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:56.382702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36804238-8a7c-46b3-a435-03534b214b38 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:56.383081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36804238-8a7c-46b3-a435-03534b214b38 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:56.383393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36804238-8a7c-46b3-a435-03534b214b38 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:56.384887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 159/316] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:03:56.391031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796ddc7-74bc-4002-be7d-ec501535c2a7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:56.391316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796ddc7-74bc-4002-be7d-ec501535c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:56.391594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796ddc7-74bc-4002-be7d-ec501535c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:56.392547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:56.411258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8796ddc7-74bc-4002-be7d-ec501535c2a7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:56.412398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8796ddc7-74bc-4002-be7d-ec501535c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:56.414525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:56.486925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 160/317] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:56.492138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 158/318] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:03:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:03:56.498335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796ddc7-74bc-4002-be7d-ec501535c2a7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:56.600854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796ddc7-74bc-4002-be7d-ec501535c2a7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:56.638700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e2d4c-56af-4eb8-b97e-da963fd713af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:56.639349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e2d4c-56af-4eb8-b97e-da963fd713af None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:56.639625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e2d4c-56af-4eb8-b97e-da963fd713af None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:56.639896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e2d4c-56af-4eb8-b97e-da963fd713af None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:56.641046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e2d4c-56af-4eb8-b97e-da963fd713af None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:56.642542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e2d4c-56af-4eb8-b97e-da963fd713af None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:56.648681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 161/319] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:03:56 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:56.654057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172fef4-eea3-4846-9dad-446897444f3c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:56.767308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e2d4c-56af-4eb8-b97e-da963fd713af None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:56.779378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:56.790802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172fef4-eea3-4846-9dad-446897444f3c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:56.826135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:56.833214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:56.834909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:56.835578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:56.836021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:56.837204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] RBAC: Adding query filter params (role.id=424ecb0deee84a068044178a8ee9ad52, scope.project.id=362cde2d45474fbcadbd8b85adbbac97, user.id=fd08d695035147fbb49f0b3364772c22) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:56.837665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:56.839689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:56.848857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 159/320] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:03:56 2022] GET /identity/v3/role_assignments?user.id=fd08d695035147fbb49f0b3364772c22&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=424ecb0deee84a068044178a8ee9ad52 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:56.905462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e2d4c-56af-4eb8-b97e-da963fd713af None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:56.947182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7d891-1752-4749-bbe3-5453cb0a4ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:56.947842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7d891-1752-4749-bbe3-5453cb0a4ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:56.948125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7d891-1752-4749-bbe3-5453cb0a4ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:56.948396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7d891-1752-4749-bbe3-5453cb0a4ae1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:56.949703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7d891-1752-4749-bbe3-5453cb0a4ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:56.951760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7d891-1752-4749-bbe3-5453cb0a4ae1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:56.953778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 162/321] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:03:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:56.959536 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:57.088238 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d6e4c-b4d5-46ce-bff4-8008abd0ad60 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:57.140851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11960a-ed4b-4869-bf33-9b5e3e372714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:57.141468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11960a-ed4b-4869-bf33-9b5e3e372714 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:57.141745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11960a-ed4b-4869-bf33-9b5e3e372714 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:57.142066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11960a-ed4b-4869-bf33-9b5e3e372714 None admin] PATH_INFO: `/v3/groups` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:57.142716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11960a-ed4b-4869-bf33-9b5e3e372714 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:57.144150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11960a-ed4b-4869-bf33-9b5e3e372714 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:57.164609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 160/322] 10.222.0.95 () {60 vars in 1134 bytes} [Wed Aug 31 19:03:56 2022] POST /identity/v3/groups => generated 226 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:57.197125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e2b7-3ece-4c6d-9e36-f9d7348c27c4 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:57.197500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e2b7-3ece-4c6d-9e36-f9d7348c27c4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:57.197778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5e2b7-3ece-4c6d-9e36-f9d7348c27c4 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:57.199300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 163/323] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:57.205695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b611-0d56-4d7e-b2da-ab31d095a429 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:57.206049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b611-0d56-4d7e-b2da-ab31d095a429 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:57.206252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b611-0d56-4d7e-b2da-ab31d095a429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:57.226633 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7f72b611-0d56-4d7e-b2da-ab31d095a429 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:57.227838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7f72b611-0d56-4d7e-b2da-ab31d095a429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:57.249614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01f2d-414e-499e-ae5f-7dec1e0a8890 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:57.250094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01f2d-414e-499e-ae5f-7dec1e0a8890 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:57.250477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01f2d-414e-499e-ae5f-7dec1e0a8890 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:57.252685 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 164/324] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:03:57.258250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b9af8-9f40-4b81-9906-b17dbc128e02 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:57.258546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b9af8-9f40-4b81-9906-b17dbc128e02 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:57.258790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b9af8-9f40-4b81-9906-b17dbc128e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:57.279132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f90b9af8-9f40-4b81-9906-b17dbc128e02 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:57.280469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f90b9af8-9f40-4b81-9906-b17dbc128e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:57.286508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:57.323888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 161/325] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:03:57.368519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 165/326] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:03:57.595812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b611-0d56-4d7e-b2da-ab31d095a429 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:57.651039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b9af8-9f40-4b81-9906-b17dbc128e02 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:57.654889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:57.730986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b611-0d56-4d7e-b2da-ab31d095a429 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:57.788117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1317a0-a8cd-4a5d-8c92-ba5f58e8a042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:57.791593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1317a0-a8cd-4a5d-8c92-ba5f58e8a042 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:57.791593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1317a0-a8cd-4a5d-8c92-ba5f58e8a042 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:57.791593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1317a0-a8cd-4a5d-8c92-ba5f58e8a042 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:57.807887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1317a0-a8cd-4a5d-8c92-ba5f58e8a042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-eb1317a0-a8cd-4a5d-8c92-ba5f58e8a042 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:57.814130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:57.815104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:03:57.815892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:03:57.816492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 162/327] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:57.821729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1317a0-a8cd-4a5d-8c92-ba5f58e8a042 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:57.822328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b9af8-9f40-4b81-9906-b17dbc128e02 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:57.863004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:57.872235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2197d22-7b4c-465a-ba1c-3389899630ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:57.873525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2197d22-7b4c-465a-ba1c-3389899630ee None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:57.873878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2197d22-7b4c-465a-ba1c-3389899630ee None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:57.874216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2197d22-7b4c-465a-ba1c-3389899630ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:57.886067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2197d22-7b4c-465a-ba1c-3389899630ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-d2197d22-7b4c-465a-ba1c-3389899630ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:03:57.892471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:57.893581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:03:57.894552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:03:57.902148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 166/328] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:03:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:57.903810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2197d22-7b4c-465a-ba1c-3389899630ee None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:57.972630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1317a0-a8cd-4a5d-8c92-ba5f58e8a042 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:58.001009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:58.012099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:58.012943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:58.013227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:58.013499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:58.014172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:58.014474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:58.015872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:58.028459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 163/329] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:58.034321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:58.046070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2197d22-7b4c-465a-ba1c-3389899630ee None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:58.087842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:58.088462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:58.088783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:58.089055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:58.089722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] RBAC: Adding query filter params (name=member) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:58.090025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:58.092939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:58.107712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 167/330] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:03:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:58.111319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:58.145820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:58.152127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597ad24-a2cb-4538-a028-c0e8c90d0f04 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:58.192964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7694e4-9695-4b43-9d5d-d766da74e829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:58.193456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7694e4-9695-4b43-9d5d-d766da74e829 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:58.193736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7694e4-9695-4b43-9d5d-d766da74e829 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:58.194018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7694e4-9695-4b43-9d5d-d766da74e829 None admin] PATH_INFO: `/v3/users/b931b49238ee4267bf80a18fb274e197` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:58.195800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7694e4-9695-4b43-9d5d-d766da74e829 None admin] RBAC: Authorizing `identity:get_user(user_id=b931b49238ee4267bf80a18fb274e197)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:58.197471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7694e4-9695-4b43-9d5d-d766da74e829 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:58.199654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 164/331] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:58 2022] GET /identity/v3/users/b931b49238ee4267bf80a18fb274e197 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:58.205991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7694e4-9695-4b43-9d5d-d766da74e829 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:58.240419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66fbae-1457-4af1-b3c5-f312114f460a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:58.276971 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:58.291547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a12616-8be5-4fdf-9da1-df5cb16d762e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:58.292769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a12616-8be5-4fdf-9da1-df5cb16d762e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:58.293200 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a12616-8be5-4fdf-9da1-df5cb16d762e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:58.294401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a12616-8be5-4fdf-9da1-df5cb16d762e None admin] PATH_INFO: `/v3/users/7197913dbfe04fddb955a30a5a90696e` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:58.297057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a12616-8be5-4fdf-9da1-df5cb16d762e None admin] RBAC: Authorizing `identity:get_user(user_id=7197913dbfe04fddb955a30a5a90696e)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:58.298697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a12616-8be5-4fdf-9da1-df5cb16d762e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:58.300791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 168/332] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:03:58 2022] GET /identity/v3/users/7197913dbfe04fddb955a30a5a90696e => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:58.307936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a12616-8be5-4fdf-9da1-df5cb16d762e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:58.335583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7694e4-9695-4b43-9d5d-d766da74e829 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:58.384505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd2879-e490-4b72-a0b7-c87f48a377cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:58.385925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd2879-e490-4b72-a0b7-c87f48a377cb None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:58.386782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd2879-e490-4b72-a0b7-c87f48a377cb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:58.387208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd2879-e490-4b72-a0b7-c87f48a377cb None admin] PATH_INFO: `/v3/system/users/b931b49238ee4267bf80a18fb274e197/roles/424ecb0deee84a068044178a8ee9ad52` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:58.390621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd2879-e490-4b72-a0b7-c87f48a377cb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b931b49238ee4267bf80a18fb274e197, role_id=424ecb0deee84a068044178a8ee9ad52)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:58.392483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd2879-e490-4b72-a0b7-c87f48a377cb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:58.406055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 165/333] 10.222.0.95 () {58 vars in 1334 bytes} [Wed Aug 31 19:03:58 2022] PUT /identity/v3/system/users/b931b49238ee4267bf80a18fb274e197/roles/424ecb0deee84a068044178a8ee9ad52 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:03:58.453470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a12616-8be5-4fdf-9da1-df5cb16d762e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:58.494315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3446b-ecd3-4504-ba26-766d598e4fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:58.495204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3446b-ecd3-4504-ba26-766d598e4fca None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:58.495656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3446b-ecd3-4504-ba26-766d598e4fca None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:58.496075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3446b-ecd3-4504-ba26-766d598e4fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:58.498421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3446b-ecd3-4504-ba26-766d598e4fca None admin] RBAC: Adding query filter params (role.id=c2bcd89a362f4fb3a73e0f8ada320802, user.id=7197913dbfe04fddb955a30a5a90696e, scope.system=all) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:58.499006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3446b-ecd3-4504-ba26-766d598e4fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:58.500831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3446b-ecd3-4504-ba26-766d598e4fca None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:58.518408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 169/334] 10.222.0.95 () {56 vars in 1308 bytes} [Wed Aug 31 19:03:58 2022] GET /identity/v3/role_assignments?user.id=7197913dbfe04fddb955a30a5a90696e&scope.system=all&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:03:59.683044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b468fad-08f6-44bf-a9ae-603325cf9da8 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:59.683570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b468fad-08f6-44bf-a9ae-603325cf9da8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:59.683570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b468fad-08f6-44bf-a9ae-603325cf9da8 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:59.685902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 166/335] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:59.691400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0cb4-d1b2-48d7-a23d-e3ff1607d972 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:59.691903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0cb4-d1b2-48d7-a23d-e3ff1607d972 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:59.691980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0cb4-d1b2-48d7-a23d-e3ff1607d972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:59.717613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-495f0cb4-d1b2-48d7-a23d-e3ff1607d972 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:59.718838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-495f0cb4-d1b2-48d7-a23d-e3ff1607d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:59.779585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:59.806370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 170/336] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:00.132697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b468fad-08f6-44bf-a9ae-603325cf9da8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:00.171984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:00.295883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b468fad-08f6-44bf-a9ae-603325cf9da8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:00.336057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32628-d27b-4f15-a2a8-184700208535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:00.336732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32628-d27b-4f15-a2a8-184700208535 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:00.337017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32628-d27b-4f15-a2a8-184700208535 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:00.337286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32628-d27b-4f15-a2a8-184700208535 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:00.352530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b32628-d27b-4f15-a2a8-184700208535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-35b32628-d27b-4f15-a2a8-184700208535 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:00.358571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:00.359490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:00.360294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:00.361601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 167/337] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:00.366570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f0cb4-d1b2-48d7-a23d-e3ff1607d972 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:00.514018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f0cb4-d1b2-48d7-a23d-e3ff1607d972 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:00.558081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9116769-aabf-4a1b-b834-27a12200cc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:00.558698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116769-aabf-4a1b-b834-27a12200cc1d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:00.558985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116769-aabf-4a1b-b834-27a12200cc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:00.559791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116769-aabf-4a1b-b834-27a12200cc1d None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:00.560654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9116769-aabf-4a1b-b834-27a12200cc1d None admin] RBAC: Adding query filter params (name=member) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:00.560830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9116769-aabf-4a1b-b834-27a12200cc1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:00.562213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9116769-aabf-4a1b-b834-27a12200cc1d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:00.577961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 171/338] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:00.583625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32628-d27b-4f15-a2a8-184700208535 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:00.750261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:00.772467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32628-d27b-4f15-a2a8-184700208535 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:00.824879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4997ac-19b7-4544-a420-57c2e4f735cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:00.827101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4997ac-19b7-4544-a420-57c2e4f735cc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:00.827634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4997ac-19b7-4544-a420-57c2e4f735cc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:00.828100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4997ac-19b7-4544-a420-57c2e4f735cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:00.895535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab22eb-5af8-4aef-bfcb-189cf3701199 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:00.896052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab22eb-5af8-4aef-bfcb-189cf3701199 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:00.896052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab22eb-5af8-4aef-bfcb-189cf3701199 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:00.897686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 172/339] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:00.912837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4997ac-19b7-4544-a420-57c2e4f735cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:00.913707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a74c-990f-449b-b13b-14e661371fbc None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:00.914033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a74c-990f-449b-b13b-14e661371fbc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:00.914306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731a74c-990f-449b-b13b-14e661371fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:00.915383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4997ac-19b7-4544-a420-57c2e4f735cc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:00.973940 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a731a74c-990f-449b-b13b-14e661371fbc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:00.974921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a731a74c-990f-449b-b13b-14e661371fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:01.061792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:01.078771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:01.081782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 168/340] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:01.086581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4997ac-19b7-4544-a420-57c2e4f735cc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:01.171430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 173/341] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:01.272348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4997ac-19b7-4544-a420-57c2e4f735cc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:01.323870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78e251-8721-4831-85f1-b7d8fbfef9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:01.324802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78e251-8721-4831-85f1-b7d8fbfef9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.325885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78e251-8721-4831-85f1-b7d8fbfef9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.326535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78e251-8721-4831-85f1-b7d8fbfef9b5 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.332807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78e251-8721-4831-85f1-b7d8fbfef9b5 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:01.334548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78e251-8721-4831-85f1-b7d8fbfef9b5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:01.336464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 169/342] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:01 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:01.338583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d6b5-e448-427a-9a62-c9aac684a695 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.338943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d6b5-e448-427a-9a62-c9aac684a695 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.339285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23d6b5-e448-427a-9a62-c9aac684a695 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.342017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 174/343] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:01.342893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78e251-8721-4831-85f1-b7d8fbfef9b5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:01.372661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33fcf7-4ede-484f-8a44-5c11cf02e64e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.373116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33fcf7-4ede-484f-8a44-5c11cf02e64e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.373389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33fcf7-4ede-484f-8a44-5c11cf02e64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.412707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-bc33fcf7-4ede-484f-8a44-5c11cf02e64e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:01.414368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-bc33fcf7-4ede-484f-8a44-5c11cf02e64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:01.445691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:01.532749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 175/344] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:01.539082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78e251-8721-4831-85f1-b7d8fbfef9b5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:01.577737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15776633-7d6f-4ed4-9d70-315e35a4824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:01.578728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776633-7d6f-4ed4-9d70-315e35a4824d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.579206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776633-7d6f-4ed4-9d70-315e35a4824d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.579671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776633-7d6f-4ed4-9d70-315e35a4824d None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.586867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15776633-7d6f-4ed4-9d70-315e35a4824d None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:01.588887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15776633-7d6f-4ed4-9d70-315e35a4824d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:01.591515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 170/345] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:01 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:01.597163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33fcf7-4ede-484f-8a44-5c11cf02e64e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:01.613998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab1665-47bd-4258-925b-b1ca314eaf93 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.614299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab1665-47bd-4258-925b-b1ca314eaf93 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.614544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab1665-47bd-4258-925b-b1ca314eaf93 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.616018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 171/346] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:01.621054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd927f5-29d3-4226-abb4-2614d743044d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.621405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd927f5-29d3-4226-abb4-2614d743044d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.621575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd927f5-29d3-4226-abb4-2614d743044d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.648207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4cd927f5-29d3-4226-abb4-2614d743044d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:01.649757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4cd927f5-29d3-4226-abb4-2614d743044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:01.775562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33fcf7-4ede-484f-8a44-5c11cf02e64e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:01.779517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:01.799478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 172/347] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:01.801562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e56a64-e79e-4d4b-9c1c-b356c36f4cca None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.801854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e56a64-e79e-4d4b-9c1c-b356c36f4cca None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.802127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e56a64-e79e-4d4b-9c1c-b356c36f4cca None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.804006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 173/348] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:01.804755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e56a64-e79e-4d4b-9c1c-b356c36f4cca None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:01.833154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62007a-1ee1-4df1-ae3d-46bcab057763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:01.833695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62007a-1ee1-4df1-ae3d-46bcab057763 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.833971 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62007a-1ee1-4df1-ae3d-46bcab057763 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.834232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62007a-1ee1-4df1-ae3d-46bcab057763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.834975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62007a-1ee1-4df1-ae3d-46bcab057763 None admin] RBAC: Adding query filter params (group.id=ca9bd4608a6f4b86834b5b7f0d766ef7, role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:01.835279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62007a-1ee1-4df1-ae3d-46bcab057763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:01.836763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62007a-1ee1-4df1-ae3d-46bcab057763 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:01.847157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 176/349] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:01 2022] GET /identity/v3/role_assignments?group.id=ca9bd4608a6f4b86834b5b7f0d766ef7&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:01.848431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eba8d-75ab-4849-919b-1ac5aadc5503 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:01.848773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eba8d-75ab-4849-919b-1ac5aadc5503 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:01.849049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eba8d-75ab-4849-919b-1ac5aadc5503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:01.871294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d75eba8d-75ab-4849-919b-1ac5aadc5503 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:01.874680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d75eba8d-75ab-4849-919b-1ac5aadc5503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:01.893734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:01.964263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 177/350] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:02.009099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75eba8d-75ab-4849-919b-1ac5aadc5503 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:02.016014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e56a64-e79e-4d4b-9c1c-b356c36f4cca None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:02.083525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c3bb-8333-4f69-b3b2-5a4baf30ae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:02.084147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c3bb-8333-4f69-b3b2-5a4baf30ae89 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:02.084431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c3bb-8333-4f69-b3b2-5a4baf30ae89 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:02.084727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c3bb-8333-4f69-b3b2-5a4baf30ae89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:02.104957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5c3bb-8333-4f69-b3b2-5a4baf30ae89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-eed5c3bb-8333-4f69-b3b2-5a4baf30ae89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:02.111501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:02.112523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:02.114464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 174/351] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:02.119293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c3bb-8333-4f69-b3b2-5a4baf30ae89 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:02.228398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:02.233109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75eba8d-75ab-4849-919b-1ac5aadc5503 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:02.266569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c3bb-8333-4f69-b3b2-5a4baf30ae89 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:02.280875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:02.283428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e61bd-5749-4e74-a609-a6315af691e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:02.284082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e61bd-5749-4e74-a609-a6315af691e4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:02.284361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e61bd-5749-4e74-a609-a6315af691e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:02.294248 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e61bd-5749-4e74-a609-a6315af691e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:02.305205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e61bd-5749-4e74-a609-a6315af691e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-0e3e61bd-5749-4e74-a609-a6315af691e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:02.314773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:02.315779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:02.317155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 178/352] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:02.317837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e61bd-5749-4e74-a609-a6315af691e4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:02.329673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:02.330755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:02.331240 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:02.331719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:02.332757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:02.333325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:02.335164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:02.347276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 175/353] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:02.348612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:02.459365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e61bd-5749-4e74-a609-a6315af691e4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:02.478400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ac270-ec4b-42a8-8edf-0f43989fef95 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:02.529027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbf2b3-c3cb-4156-9f25-d85f727d435b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:02.530810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf2b3-c3cb-4156-9f25-d85f727d435b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:02.531950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf2b3-c3cb-4156-9f25-d85f727d435b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:02.533022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf2b3-c3cb-4156-9f25-d85f727d435b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:02.541949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:02.542396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:02.542785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:02.543245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:02.544682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:02.545266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:02.547717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:02.559586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbf2b3-c3cb-4156-9f25-d85f727d435b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:02.564300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 176/354] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:02.565816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-5adbf2b3-c3cb-4156-9f25-d85f727d435b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:02.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:02.572159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:02.573761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:02.582577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 179/355] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:02.584372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbf2b3-c3cb-4156-9f25-d85f727d435b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:02.625396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:02.636047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:02.752782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351435e-947e-4f92-9e1a-ff08b3c8c582 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:02.760058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbf2b3-c3cb-4156-9f25-d85f727d435b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:02.805273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a96289-33c2-49f5-90e5-133cf2b46267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:02.806618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96289-33c2-49f5-90e5-133cf2b46267 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:02.807763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96289-33c2-49f5-90e5-133cf2b46267 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:02.808245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a96289-33c2-49f5-90e5-133cf2b46267 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:02.825516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a96289-33c2-49f5-90e5-133cf2b46267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:02.826683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae48f62-f29f-4615-9d4e-1bd4e3f576ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:02.827307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48f62-f29f-4615-9d4e-1bd4e3f576ac None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:02.827595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48f62-f29f-4615-9d4e-1bd4e3f576ac None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:02.827853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48f62-f29f-4615-9d4e-1bd4e3f576ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-07a96289-33c2-49f5-90e5-133cf2b46267 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:02.834079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:02.835054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 19:04:02.836011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:02.838325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 177/356] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:02.839190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a96289-33c2-49f5-90e5-133cf2b46267 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:02.925727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae48f62-f29f-4615-9d4e-1bd4e3f576ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:02.927239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae48f62-f29f-4615-9d4e-1bd4e3f576ac None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:02.957731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:02.965375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:02.979943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a96289-33c2-49f5-90e5-133cf2b46267 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:03.030942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba074a9-9dfc-4888-bee1-d06344696315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:03.031568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba074a9-9dfc-4888-bee1-d06344696315 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.031852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba074a9-9dfc-4888-bee1-d06344696315 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.032185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba074a9-9dfc-4888-bee1-d06344696315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.091320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 180/357] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:03.092291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae48f62-f29f-4615-9d4e-1bd4e3f576ac None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:03.120354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba074a9-9dfc-4888-bee1-d06344696315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:03.122913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba074a9-9dfc-4888-bee1-d06344696315 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:03.235382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae48f62-f29f-4615-9d4e-1bd4e3f576ac None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:03.258048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:03.281801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:03.282483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.282763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.283033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.284349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:03.284349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:03.285570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:03.287863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:03.295287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 178/358] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:03.295958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 181/359] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:03.296679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55e0b-06f9-43cd-b891-f6f17e8958ef None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.297096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55e0b-06f9-43cd-b891-f6f17e8958ef None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.297377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55e0b-06f9-43cd-b891-f6f17e8958ef None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.298800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 179/360] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:03.299382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55e0b-06f9-43cd-b891-f6f17e8958ef None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:03.300952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:03.434749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3c33-3b56-4a4a-adda-a235add2f59c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:03.446624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55e0b-06f9-43cd-b891-f6f17e8958ef None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:03.478351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:03.478964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.479241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.479545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.480409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:03.480524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:03.482641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:03.490355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce215ac-ad68-4738-a3bb-3139a031badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:03.490922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce215ac-ad68-4738-a3bb-3139a031badc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.491205 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce215ac-ad68-4738-a3bb-3139a031badc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.491476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce215ac-ad68-4738-a3bb-3139a031badc None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.492692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 182/361] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:03.493354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce215ac-ad68-4738-a3bb-3139a031badc None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:03.493551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:03.494869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce215ac-ad68-4738-a3bb-3139a031badc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:03.496814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 180/362] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:03 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:03.498176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742fa5f-be6c-409c-86e6-e7b342e25a4e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.498460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742fa5f-be6c-409c-86e6-e7b342e25a4e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.498732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742fa5f-be6c-409c-86e6-e7b342e25a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.525491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7742fa5f-be6c-409c-86e6-e7b342e25a4e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:03.527635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7742fa5f-be6c-409c-86e6-e7b342e25a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:03.546329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:03.607360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:03.614582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f04b39-1129-444c-84c2-9f0248095fb1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:03.619566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 181/363] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:03.620290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742fa5f-be6c-409c-86e6-e7b342e25a4e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:03.664360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e54a7-c6f9-4a11-9f6b-58ef539287a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:03.665644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e54a7-c6f9-4a11-9f6b-58ef539287a2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.666138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e54a7-c6f9-4a11-9f6b-58ef539287a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.666853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e54a7-c6f9-4a11-9f6b-58ef539287a2 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.668362 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e54a7-c6f9-4a11-9f6b-58ef539287a2 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:03.671558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e54a7-c6f9-4a11-9f6b-58ef539287a2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:03.674305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 183/364] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:03 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:03.675553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e54a7-c6f9-4a11-9f6b-58ef539287a2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:03.735686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742fa5f-be6c-409c-86e6-e7b342e25a4e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:03.774756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f20cb8-4a16-48c5-b32e-5590e9240373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:03.775365 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f20cb8-4a16-48c5-b32e-5590e9240373 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.775710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f20cb8-4a16-48c5-b32e-5590e9240373 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.776069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f20cb8-4a16-48c5-b32e-5590e9240373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.814284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e54a7-c6f9-4a11-9f6b-58ef539287a2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:03.828012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:03.852819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f20cb8-4a16-48c5-b32e-5590e9240373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:03.854391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f20cb8-4a16-48c5-b32e-5590e9240373 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:03.862485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476167e7-8069-442e-b868-cd95536f6a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:03.863435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476167e7-8069-442e-b868-cd95536f6a5a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:03.864371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476167e7-8069-442e-b868-cd95536f6a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:03.864854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476167e7-8069-442e-b868-cd95536f6a5a None admin] PATH_INFO: `/v3/users/b931b49238ee4267bf80a18fb274e197` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:03.866285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476167e7-8069-442e-b868-cd95536f6a5a None admin] RBAC: Authorizing `identity:get_user(user_id=b931b49238ee4267bf80a18fb274e197)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:03.868711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476167e7-8069-442e-b868-cd95536f6a5a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:03.871112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 184/365] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:04:03 2022] GET /identity/v3/users/b931b49238ee4267bf80a18fb274e197 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:03.876724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476167e7-8069-442e-b868-cd95536f6a5a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:03.876960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:04.016415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 182/366] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:04.017627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f20cb8-4a16-48c5-b32e-5590e9240373 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.018068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476167e7-8069-442e-b868-cd95536f6a5a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.063343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485f388-e7d6-4dae-8c2b-dd6849c5df1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.063906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485f388-e7d6-4dae-8c2b-dd6849c5df1b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.064192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485f388-e7d6-4dae-8c2b-dd6849c5df1b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.064469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485f388-e7d6-4dae-8c2b-dd6849c5df1b None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.065662 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485f388-e7d6-4dae-8c2b-dd6849c5df1b None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:04.067118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485f388-e7d6-4dae-8c2b-dd6849c5df1b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:04.068724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 185/367] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:03 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:04.069485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485f388-e7d6-4dae-8c2b-dd6849c5df1b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.132968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:04.133941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:04.149790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f20cb8-4a16-48c5-b32e-5590e9240373 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.196365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485f388-e7d6-4dae-8c2b-dd6849c5df1b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.201218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4fc5-7491-4b92-bee9-f8360488be2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.201218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4fc5-7491-4b92-bee9-f8360488be2f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.201673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4fc5-7491-4b92-bee9-f8360488be2f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.201799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4fc5-7491-4b92-bee9-f8360488be2f None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.203045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d4fc5-7491-4b92-bee9-f8360488be2f None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:04.205013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d4fc5-7491-4b92-bee9-f8360488be2f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:04.206637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 183/368] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:04.207429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4fc5-7491-4b92-bee9-f8360488be2f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.247217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.247838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.248106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.248381 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.250274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] RBAC: Adding query filter params (role.id=424ecb0deee84a068044178a8ee9ad52, user.id=b931b49238ee4267bf80a18fb274e197, scope.system=all) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:04.250908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:04.252664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:04.273100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 186/369] 10.222.0.95 () {56 vars in 1308 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/role_assignments?user.id=b931b49238ee4267bf80a18fb274e197&scope.system=all&role.id=424ecb0deee84a068044178a8ee9ad52 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:04.274031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.345893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4fc5-7491-4b92-bee9-f8360488be2f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.400479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69549e3c-c39f-4c8a-80b3-cf14082d0f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.401190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69549e3c-c39f-4c8a-80b3-cf14082d0f84 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.401474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69549e3c-c39f-4c8a-80b3-cf14082d0f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.401745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69549e3c-c39f-4c8a-80b3-cf14082d0f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.418040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69549e3c-c39f-4c8a-80b3-cf14082d0f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-69549e3c-c39f-4c8a-80b3-cf14082d0f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:04.425146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:04.426211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:04.427187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:04.428171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 184/370] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:04.429069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69549e3c-c39f-4c8a-80b3-cf14082d0f84 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.440759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee11ef6-ad2c-4a53-a783-1f70a9596b71 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.444300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:04.467292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:04.500290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4e05e-a5a8-4ca4-90a1-bc7fa5936f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.501692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4e05e-a5a8-4ca4-90a1-bc7fa5936f2b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.501967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4e05e-a5a8-4ca4-90a1-bc7fa5936f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.502246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4e05e-a5a8-4ca4-90a1-bc7fa5936f2b None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.503372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4e05e-a5a8-4ca4-90a1-bc7fa5936f2b None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:04.504864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4e05e-a5a8-4ca4-90a1-bc7fa5936f2b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:04.506532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 187/371] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:04.507298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4e05e-a5a8-4ca4-90a1-bc7fa5936f2b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.563379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69549e3c-c39f-4c8a-80b3-cf14082d0f84 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.606424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcfd99-ab83-459a-921d-3a40f891475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.606964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcfd99-ab83-459a-921d-3a40f891475a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.607259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcfd99-ab83-459a-921d-3a40f891475a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.607523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcfd99-ab83-459a-921d-3a40f891475a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.608248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcfd99-ab83-459a-921d-3a40f891475a None admin] RBAC: Adding query filter params (group.id=ca9bd4608a6f4b86834b5b7f0d766ef7, role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:04.608552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcfd99-ab83-459a-921d-3a40f891475a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:04.610033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcfd99-ab83-459a-921d-3a40f891475a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:04.615723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 185/372] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/role_assignments?group.id=ca9bd4608a6f4b86834b5b7f0d766ef7&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=79dbf286cf884619af62f33ea75c52bf => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:04.620620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8459916-bcb2-41dd-884b-1ee3bb312c4a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.620620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8459916-bcb2-41dd-884b-1ee3bb312c4a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.620620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8459916-bcb2-41dd-884b-1ee3bb312c4a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.620620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 186/373] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:04.620620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8459916-bcb2-41dd-884b-1ee3bb312c4a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.640931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4e05e-a5a8-4ca4-90a1-bc7fa5936f2b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.690088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.691313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.691609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.691887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.693499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] RBAC: Adding query filter params (group.id=ca9bd4608a6f4b86834b5b7f0d766ef7, role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=362cde2d45474fbcadbd8b85adbbac97) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:04.693499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:04.694522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:04.702528 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 188/374] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/role_assignments?group.id=ca9bd4608a6f4b86834b5b7f0d766ef7&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:04.703217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.746086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:04.757006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:04.772630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8459916-bcb2-41dd-884b-1ee3bb312c4a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.827094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d62c59-2b54-4cb8-bf45-c5b19db52768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.828001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d62c59-2b54-4cb8-bf45-c5b19db52768 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.829053 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d62c59-2b54-4cb8-bf45-c5b19db52768 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.829508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d62c59-2b54-4cb8-bf45-c5b19db52768 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.830350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d62c59-2b54-4cb8-bf45-c5b19db52768 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:04.830981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d62c59-2b54-4cb8-bf45-c5b19db52768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:04.832731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d62c59-2b54-4cb8-bf45-c5b19db52768 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:04.844021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc768a1-f2f5-4637-bd40-8297d1227457 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:04.845655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 187/375] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:04.848324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d6dc-1ad6-4c8c-b6f9-640a0171fd8c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.848795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d6dc-1ad6-4c8c-b6f9-640a0171fd8c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.849211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657d6dc-1ad6-4c8c-b6f9-640a0171fd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.876091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6657d6dc-1ad6-4c8c-b6f9-640a0171fd8c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:04.878340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6657d6dc-1ad6-4c8c-b6f9-640a0171fd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:04.899073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84559d80-35df-4248-9a7d-73217491b93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:04.899722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84559d80-35df-4248-9a7d-73217491b93b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:04.900006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84559d80-35df-4248-9a7d-73217491b93b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:04.900277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84559d80-35df-4248-9a7d-73217491b93b None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:04.901478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84559d80-35df-4248-9a7d-73217491b93b None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:04.902980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84559d80-35df-4248-9a7d-73217491b93b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:04.905981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 189/376] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:04.912420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84559d80-35df-4248-9a7d-73217491b93b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:04.972828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 188/377] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:04.974068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d6dc-1ad6-4c8c-b6f9-640a0171fd8c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:05.044934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84559d80-35df-4248-9a7d-73217491b93b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:05.048009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:05.116969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c4b69-4975-426c-b8cc-9d3920b01641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:05.118898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c4b69-4975-426c-b8cc-9d3920b01641 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:05.122572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c4b69-4975-426c-b8cc-9d3920b01641 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:05.123235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c4b69-4975-426c-b8cc-9d3920b01641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:05.141340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:05.166688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657d6dc-1ad6-4c8c-b6f9-640a0171fd8c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:05.236588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:05.237213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:05.237494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:05.237760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:05.238492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] RBAC: Adding query filter params (group.id=ca9bd4608a6f4b86834b5b7f0d766ef7, role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=362cde2d45474fbcadbd8b85adbbac97) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:05.241300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:05.242779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:05.258213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 189/378] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/role_assignments?group.id=ca9bd4608a6f4b86834b5b7f0d766ef7&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=79dbf286cf884619af62f33ea75c52bf => generated 260 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:05.268805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c4b69-4975-426c-b8cc-9d3920b01641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:05.270328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c4b69-4975-426c-b8cc-9d3920b01641 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:05.334463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:05.436515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 190/379] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:05.442603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c4b69-4975-426c-b8cc-9d3920b01641 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:05.448893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:05.538328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545004da-96a5-42ad-a2dc-1895aeb07def None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:05.576387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438e930-ef00-45a2-ad10-131f2714bb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:05.577053 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438e930-ef00-45a2-ad10-131f2714bb5c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:05.577336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438e930-ef00-45a2-ad10-131f2714bb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:05.577609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438e930-ef00-45a2-ad10-131f2714bb5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:05.588938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:05.590185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438e930-ef00-45a2-ad10-131f2714bb5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8438e930-ef00-45a2-ad10-131f2714bb5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:05.596439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:05.597498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:05.598290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:05.599090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 190/380] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:05.612096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438e930-ef00-45a2-ad10-131f2714bb5c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:05.660397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c4b69-4975-426c-b8cc-9d3920b01641 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:05.729033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17adcd-3bc1-4bff-a5e8-158b49ae1f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:05.729733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17adcd-3bc1-4bff-a5e8-158b49ae1f21 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:05.730020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17adcd-3bc1-4bff-a5e8-158b49ae1f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:05.730346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17adcd-3bc1-4bff-a5e8-158b49ae1f21 None admin] PATH_INFO: `/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:05.732386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17adcd-3bc1-4bff-a5e8-158b49ae1f21 None admin] RBAC: Authorizing `identity:get_project(project_id=4b2aa0a8faf9471ebd699445da59c0dc)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:05.734142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17adcd-3bc1-4bff-a5e8-158b49ae1f21 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:05.735879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 191/381] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:05 2022] GET /identity/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc => generated 344 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:05.744595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17adcd-3bc1-4bff-a5e8-158b49ae1f21 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:05.820374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438e930-ef00-45a2-ad10-131f2714bb5c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:05.821620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:05.855896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17adcd-3bc1-4bff-a5e8-158b49ae1f21 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:05.880308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:05.882195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:05.882969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:05.883445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:05.884586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:05.885526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:05.888148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:05.899341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 191/382] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:05.905414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2cb31-0083-4f03-9d8b-36f5ce7ad023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:05.906072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2cb31-0083-4f03-9d8b-36f5ce7ad023 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:05.906356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2cb31-0083-4f03-9d8b-36f5ce7ad023 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:05.906628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2cb31-0083-4f03-9d8b-36f5ce7ad023 None admin] PATH_INFO: `/v3/groups/65a7482163dd46d9914ae19c6b323846` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:05.909486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:05.914939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2cb31-0083-4f03-9d8b-36f5ce7ad023 None admin] RBAC: Authorizing `identity:get_group(group_id=65a7482163dd46d9914ae19c6b323846)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:05.916599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2cb31-0083-4f03-9d8b-36f5ce7ad023 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:05.919052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 192/383] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:05 2022] GET /identity/v3/groups/65a7482163dd46d9914ae19c6b323846 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:05.925601 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2cb31-0083-4f03-9d8b-36f5ce7ad023 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:06.041224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:06.087150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2cb31-0083-4f03-9d8b-36f5ce7ad023 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:06.105456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13d71c-da25-4847-880f-a688803d6d7d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:06.130664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:06.131537 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:06.132033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:06.132501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:06.133482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] RBAC: Adding query filter params (group.id=65a7482163dd46d9914ae19c6b323846, role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=4b2aa0a8faf9471ebd699445da59c0dc) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:06.133988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:06.135853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:06.139485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:06.145765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 193/384] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:05 2022] GET /identity/v3/role_assignments?group.id=65a7482163dd46d9914ae19c6b323846&scope.project.id=4b2aa0a8faf9471ebd699445da59c0dc&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:06.170759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e68b75-d412-4e43-8d2f-0d32467fb212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:06.171769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68b75-d412-4e43-8d2f-0d32467fb212 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:06.172273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68b75-d412-4e43-8d2f-0d32467fb212 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:06.172797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68b75-d412-4e43-8d2f-0d32467fb212 None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:06.174233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e68b75-d412-4e43-8d2f-0d32467fb212 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:06.176163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e68b75-d412-4e43-8d2f-0d32467fb212 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:06.178519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 192/385] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:05 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:06.185850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:06.313444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dd5f3-eed5-4525-aef0-8cafc332edb1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:06.352350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336e370-8887-4c3c-9f35-f461e37e14df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:06.353595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e370-8887-4c3c-9f35-f461e37e14df None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:06.354084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e370-8887-4c3c-9f35-f461e37e14df None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:06.354577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336e370-8887-4c3c-9f35-f461e37e14df None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:06.356172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336e370-8887-4c3c-9f35-f461e37e14df None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:06.358643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336e370-8887-4c3c-9f35-f461e37e14df None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:06.361040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 194/386] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:06 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:06.369958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e68b75-d412-4e43-8d2f-0d32467fb212 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:06.506679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e68b75-d412-4e43-8d2f-0d32467fb212 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:06.541359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:06.551515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5c052-c92c-4a36-a70a-b3ef63a490ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:06.552212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5c052-c92c-4a36-a70a-b3ef63a490ed None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:06.552493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5c052-c92c-4a36-a70a-b3ef63a490ed None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:06.552810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5c052-c92c-4a36-a70a-b3ef63a490ed None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/groups/ca9bd4608a6f4b86834b5b7f0d766ef7/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:06.554854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5c052-c92c-4a36-a70a-b3ef63a490ed None admin] RBAC: Authorizing `identity:create_grant(project_id=8f379c8467d44bcbbf9cbc6aac3a3231, group_id=ca9bd4608a6f4b86834b5b7f0d766ef7, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:06.556370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5c052-c92c-4a36-a70a-b3ef63a490ed None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:06.574998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 193/387] 10.222.0.95 () {58 vars in 1442 bytes} [Wed Aug 31 19:04:06 2022] PUT /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/groups/ca9bd4608a6f4b86834b5b7f0d766ef7/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:06.969579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd534b-5d00-4b30-abf7-1251dcea6cf0 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:06.969935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd534b-5d00-4b30-abf7-1251dcea6cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:06.970128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd534b-5d00-4b30-abf7-1251dcea6cf0 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:06.971723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 195/388] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:06.977017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4bf3a-e29f-4b7f-9cf0-973519866cc8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:06.977485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4bf3a-e29f-4b7f-9cf0-973519866cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:06.977560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4bf3a-e29f-4b7f-9cf0-973519866cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.001752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ebf4bf3a-e29f-4b7f-9cf0-973519866cc8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:07.002950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ebf4bf3a-e29f-4b7f-9cf0-973519866cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:07.011694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5893012-d925-45fe-82d0-fdd3488611a5 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:07.012975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5893012-d925-45fe-82d0-fdd3488611a5 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:07.013507 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5893012-d925-45fe-82d0-fdd3488611a5 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.015339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 196/389] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:07.021064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88231ecb-cc28-483b-be6c-a63d024766d9 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:07.021602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88231ecb-cc28-483b-be6c-a63d024766d9 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:07.022087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88231ecb-cc28-483b-be6c-a63d024766d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.041281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-88231ecb-cc28-483b-be6c-a63d024766d9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:07.042896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-88231ecb-cc28-483b-be6c-a63d024766d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:07.119043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 194/390] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:07.129039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:07.131631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 197/391] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:07.440620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4bf3a-e29f-4b7f-9cf0-973519866cc8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:07.472387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88231ecb-cc28-483b-be6c-a63d024766d9 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:07.593822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88231ecb-cc28-483b-be6c-a63d024766d9 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:07.609213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4bf3a-e29f-4b7f-9cf0-973519866cc8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:07.633420 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:07.636801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2ec7f-cccb-4fc6-a22d-2c26809c0897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:07.638913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2ec7f-cccb-4fc6-a22d-2c26809c0897 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:07.640076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2ec7f-cccb-4fc6-a22d-2c26809c0897 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:07.641050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2ec7f-cccb-4fc6-a22d-2c26809c0897 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.652660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661907e-c0f8-4da7-8521-dec298e3a9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:07.654230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661907e-c0f8-4da7-8521-dec298e3a9cb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:07.654582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661907e-c0f8-4da7-8521-dec298e3a9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:07.654923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661907e-c0f8-4da7-8521-dec298e3a9cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.656887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2ec7f-cccb-4fc6-a22d-2c26809c0897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-54e2ec7f-cccb-4fc6-a22d-2c26809c0897 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:07.663343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:07.664224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:07.665070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:07.666913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661907e-c0f8-4da7-8521-dec298e3a9cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:07.672291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 198/392] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4661907e-c0f8-4da7-8521-dec298e3a9cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:07.672426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:07.673520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:07.674513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:07.676883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cfb11-93d2-4545-92ec-6b6b39e499ca None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:07.677319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cfb11-93d2-4545-92ec-6b6b39e499ca None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:07.678182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cfb11-93d2-4545-92ec-6b6b39e499ca None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.680135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 199/394] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:07.680888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 195/394] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:07.681410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cfb11-93d2-4545-92ec-6b6b39e499ca None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:07.685737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661907e-c0f8-4da7-8521-dec298e3a9cb None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:07.768484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:07.814350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661907e-c0f8-4da7-8521-dec298e3a9cb None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:07.816048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cfb11-93d2-4545-92ec-6b6b39e499ca None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:07.852495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae23728-ff85-4685-9f76-1724eeda499c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:07.853297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:07.853625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae23728-ff85-4685-9f76-1724eeda499c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:07.854106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:07.854262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae23728-ff85-4685-9f76-1724eeda499c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:07.854566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:07.854811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae23728-ff85-4685-9f76-1724eeda499c None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.855216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.855783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae23728-ff85-4685-9f76-1724eeda499c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:07.856330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae23728-ff85-4685-9f76-1724eeda499c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:07.856560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] RBAC: Adding query filter params (name=member) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:07.856989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:07.858190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae23728-ff85-4685-9f76-1724eeda499c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:07.858562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:07.872275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 196/395] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:07.872643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 200/396] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:07.873742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:07.874159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055099f0-7012-4e5a-954a-08ed49f98c57 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:07.874598 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055099f0-7012-4e5a-954a-08ed49f98c57 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:07.875023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055099f0-7012-4e5a-954a-08ed49f98c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:07.895775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-055099f0-7012-4e5a-954a-08ed49f98c57 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:07.897328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-055099f0-7012-4e5a-954a-08ed49f98c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:07.937462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:07.973796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 197/397] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:07.975119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055099f0-7012-4e5a-954a-08ed49f98c57 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:08.000505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd0027-8bbe-4f19-8c4d-83591825f705 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:08.054396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:08.089851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055099f0-7012-4e5a-954a-08ed49f98c57 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:08.098226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08a5ec-bc3c-457a-bca5-05c08f7a227d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:08.098946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08a5ec-bc3c-457a-bca5-05c08f7a227d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.099232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08a5ec-bc3c-457a-bca5-05c08f7a227d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.099516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08a5ec-bc3c-457a-bca5-05c08f7a227d None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.100817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08a5ec-bc3c-457a-bca5-05c08f7a227d None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:08.102356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08a5ec-bc3c-457a-bca5-05c08f7a227d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:08.112752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 201/398] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:07 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:08.113707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08a5ec-bc3c-457a-bca5-05c08f7a227d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:08.139793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a240e-ae49-4154-b228-890f3013f5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:08.143386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a240e-ae49-4154-b228-890f3013f5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.144798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a240e-ae49-4154-b228-890f3013f5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.145278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a240e-ae49-4154-b228-890f3013f5c7 None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.147590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a240e-ae49-4154-b228-890f3013f5c7 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:08.149990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a240e-ae49-4154-b228-890f3013f5c7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:08.161554 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 198/399] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:07 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:08.164436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a240e-ae49-4154-b228-890f3013f5c7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:08.211158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:08.279804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a240e-ae49-4154-b228-890f3013f5c7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:08.319090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25855b76-ec39-4bae-88fb-9b49729d015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:08.320066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25855b76-ec39-4bae-88fb-9b49729d015d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.320853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25855b76-ec39-4bae-88fb-9b49729d015d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.322459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25855b76-ec39-4bae-88fb-9b49729d015d None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.323774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25855b76-ec39-4bae-88fb-9b49729d015d None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:08.326098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25855b76-ec39-4bae-88fb-9b49729d015d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:08.332769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 199/400] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:08 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:08.333686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25855b76-ec39-4bae-88fb-9b49729d015d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:08.350407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08a5ec-bc3c-457a-bca5-05c08f7a227d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:08.387035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa235-53a9-4f2e-aac8-030269470003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:08.388851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa235-53a9-4f2e-aac8-030269470003 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.389348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa235-53a9-4f2e-aac8-030269470003 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.389844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205aa235-53a9-4f2e-aac8-030269470003 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.391287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205aa235-53a9-4f2e-aac8-030269470003 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:08.393022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205aa235-53a9-4f2e-aac8-030269470003 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:08.395055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 202/401] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:08 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:08.396106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa235-53a9-4f2e-aac8-030269470003 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:08.448941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25855b76-ec39-4bae-88fb-9b49729d015d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:08.458945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:08.460579 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:08.491505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a413d7-117c-421f-85e2-d0f430092f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:08.492086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a413d7-117c-421f-85e2-d0f430092f55 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.492365 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a413d7-117c-421f-85e2-d0f430092f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.492662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a413d7-117c-421f-85e2-d0f430092f55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.505400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a413d7-117c-421f-85e2-d0f430092f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-53a413d7-117c-421f-85e2-d0f430092f55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:08.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:08.511787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:08.512945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 200/402] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:08.513720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a413d7-117c-421f-85e2-d0f430092f55 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:08.521539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205aa235-53a9-4f2e-aac8-030269470003 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:08.567220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7508c-dc5a-4f33-b98a-b4db5f615861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:08.567932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7508c-dc5a-4f33-b98a-b4db5f615861 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.568216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7508c-dc5a-4f33-b98a-b4db5f615861 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.568491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7508c-dc5a-4f33-b98a-b4db5f615861 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97/groups/ca9bd4608a6f4b86834b5b7f0d766ef7/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.570607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7508c-dc5a-4f33-b98a-b4db5f615861 None admin] RBAC: Authorizing `identity:create_grant(project_id=362cde2d45474fbcadbd8b85adbbac97, group_id=ca9bd4608a6f4b86834b5b7f0d766ef7, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:08.572021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7508c-dc5a-4f33-b98a-b4db5f615861 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:08.596668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 203/403] 10.222.0.95 () {58 vars in 1442 bytes} [Wed Aug 31 19:04:08 2022] PUT /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97/groups/ca9bd4608a6f4b86834b5b7f0d766ef7/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:08.598160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7508c-dc5a-4f33-b98a-b4db5f615861 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:08.698392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a413d7-117c-421f-85e2-d0f430092f55 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:08.772998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9610e61-4239-4e79-86dd-89456278c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:08.776606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9610e61-4239-4e79-86dd-89456278c2e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.776606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9610e61-4239-4e79-86dd-89456278c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.776606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9610e61-4239-4e79-86dd-89456278c2e1 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/groups/ca9bd4608a6f4b86834b5b7f0d766ef7/roles/79dbf286cf884619af62f33ea75c52bf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.777521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9610e61-4239-4e79-86dd-89456278c2e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f379c8467d44bcbbf9cbc6aac3a3231, group_id=ca9bd4608a6f4b86834b5b7f0d766ef7, role_id=79dbf286cf884619af62f33ea75c52bf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:08.779913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9610e61-4239-4e79-86dd-89456278c2e1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:08.797335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 201/404] 10.222.0.95 () {58 vars in 1442 bytes} [Wed Aug 31 19:04:08 2022] PUT /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231/groups/ca9bd4608a6f4b86834b5b7f0d766ef7/roles/79dbf286cf884619af62f33ea75c52bf => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:08.798946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5fc1d-30d2-4adc-8a1a-ade75aabbbb7 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.799247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5fc1d-30d2-4adc-8a1a-ade75aabbbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.799518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5fc1d-30d2-4adc-8a1a-ade75aabbbb7 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.804665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 202/405] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:08.808021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cf9c3-0fbb-4dcc-8fc0-d485d545216d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.808317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cf9c3-0fbb-4dcc-8fc0-d485d545216d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.808621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cf9c3-0fbb-4dcc-8fc0-d485d545216d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.810942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7508c-dc5a-4f33-b98a-b4db5f615861 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:08.833122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:08.858346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4e0cf9c3-0fbb-4dcc-8fc0-d485d545216d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:08.860927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4e0cf9c3-0fbb-4dcc-8fc0-d485d545216d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:08.878792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:08.879797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:08.880226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:08.880659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:08.881549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:08.882043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:08.886640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:08.907432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 204/406] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:08.914157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:08.922463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:09.014555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 203/407] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:09.040219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3efdad-2764-4b9d-8f8d-16990dc82bb8 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:09.040529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3efdad-2764-4b9d-8f8d-16990dc82bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:09.040815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3efdad-2764-4b9d-8f8d-16990dc82bb8 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:09.042426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 204/408] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:09.045458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3d2c7-9e6d-4fdd-9ef7-99ac200f8d62 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:09.048148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc045b-cc4d-41f3-9822-8c954b400b39 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:09.048427 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc045b-cc4d-41f3-9822-8c954b400b39 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:09.048820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc045b-cc4d-41f3-9822-8c954b400b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:09.068143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ecbc045b-cc4d-41f3-9822-8c954b400b39 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:09.069642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ecbc045b-cc4d-41f3-9822-8c954b400b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:09.090107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0ebcd-28bb-42f5-9423-1e80b68af3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:09.090811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0ebcd-28bb-42f5-9423-1e80b68af3fc None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:09.091370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0ebcd-28bb-42f5-9423-1e80b68af3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:09.091659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0ebcd-28bb-42f5-9423-1e80b68af3fc None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:09.093807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0ebcd-28bb-42f5-9423-1e80b68af3fc None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:09.095276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0ebcd-28bb-42f5-9423-1e80b68af3fc None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:09.097802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 205/409] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:08 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:09.104491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0ebcd-28bb-42f5-9423-1e80b68af3fc None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:09.157632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 205/410] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:09.180934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:09.284371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0ebcd-28bb-42f5-9423-1e80b68af3fc None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:09.334594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5a81a-9efb-4448-bfee-ec2a59e50e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:09.335227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5a81a-9efb-4448-bfee-ec2a59e50e68 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:09.335494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5a81a-9efb-4448-bfee-ec2a59e50e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:09.335764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5a81a-9efb-4448-bfee-ec2a59e50e68 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:09.336977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5a81a-9efb-4448-bfee-ec2a59e50e68 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:09.338417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5a81a-9efb-4448-bfee-ec2a59e50e68 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:09.342291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 206/411] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:09 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:09.350010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc045b-cc4d-41f3-9822-8c954b400b39 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:09.449090 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5a81a-9efb-4448-bfee-ec2a59e50e68 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:09.524745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:09.545542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc045b-cc4d-41f3-9822-8c954b400b39 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:09.602890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb145dff-3f40-45f7-b3aa-9f53ded70c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:09.603479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb145dff-3f40-45f7-b3aa-9f53ded70c5e None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:09.603754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb145dff-3f40-45f7-b3aa-9f53ded70c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:09.604023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb145dff-3f40-45f7-b3aa-9f53ded70c5e None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97/groups/ca9bd4608a6f4b86834b5b7f0d766ef7/roles/79dbf286cf884619af62f33ea75c52bf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:09.607168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb145dff-3f40-45f7-b3aa-9f53ded70c5e None admin] RBAC: Authorizing `identity:create_grant(project_id=362cde2d45474fbcadbd8b85adbbac97, group_id=ca9bd4608a6f4b86834b5b7f0d766ef7, role_id=79dbf286cf884619af62f33ea75c52bf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:09.608956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb145dff-3f40-45f7-b3aa-9f53ded70c5e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:09.619978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5a81a-9efb-4448-bfee-ec2a59e50e68 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:09.626399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 206/412] 10.222.0.95 () {58 vars in 1442 bytes} [Wed Aug 31 19:04:09 2022] PUT /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97/groups/ca9bd4608a6f4b86834b5b7f0d766ef7/roles/79dbf286cf884619af62f33ea75c52bf => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:09.627303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb145dff-3f40-45f7-b3aa-9f53ded70c5e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:09.659568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:09.702030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dff6b-e331-4f4e-998e-85c7538e5742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:09.703106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dff6b-e331-4f4e-998e-85c7538e5742 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:09.703585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dff6b-e331-4f4e-998e-85c7538e5742 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:09.704082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dff6b-e331-4f4e-998e-85c7538e5742 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:09.717151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dff6b-e331-4f4e-998e-85c7538e5742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-289dff6b-e331-4f4e-998e-85c7538e5742 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:09.723421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:09.724401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:09.725236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:09.728018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 207/413] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:09.740679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dff6b-e331-4f4e-998e-85c7538e5742 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:09.850869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb145dff-3f40-45f7-b3aa-9f53ded70c5e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:09.888452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dff6b-e331-4f4e-998e-85c7538e5742 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:09.897884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ee4ae-1afa-498d-a725-b032eb400048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:09.898720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ee4ae-1afa-498d-a725-b032eb400048 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:09.899003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ee4ae-1afa-498d-a725-b032eb400048 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:09.899320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ee4ae-1afa-498d-a725-b032eb400048 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:09.920701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ee4ae-1afa-498d-a725-b032eb400048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-758ee4ae-1afa-498d-a725-b032eb400048 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:09.923662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:09.924552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:09.925675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 207/414] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:09.940401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ee4ae-1afa-498d-a725-b032eb400048 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:09.942580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:09.943407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:09.943868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:09.944400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:09.945290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:09.945851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:09.947564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:09.956926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 208/415] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:09.960210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:09.963363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:10.022314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:10.075688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c883c-89e2-40df-b850-43ba302376c2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:10.115037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54155dd4-47a9-49c0-89b1-437bd711374a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:10.116095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54155dd4-47a9-49c0-89b1-437bd711374a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:10.116543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54155dd4-47a9-49c0-89b1-437bd711374a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:10.116996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54155dd4-47a9-49c0-89b1-437bd711374a None admin] PATH_INFO: `/v3/groups/65a7482163dd46d9914ae19c6b323846` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:10.118420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54155dd4-47a9-49c0-89b1-437bd711374a None admin] RBAC: Authorizing `identity:get_group(group_id=65a7482163dd46d9914ae19c6b323846)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:10.120113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54155dd4-47a9-49c0-89b1-437bd711374a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:10.122272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 209/416] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:09 2022] GET /identity/v3/groups/65a7482163dd46d9914ae19c6b323846 => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:10.128342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54155dd4-47a9-49c0-89b1-437bd711374a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:10.204888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ee4ae-1afa-498d-a725-b032eb400048 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:10.247118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54155dd4-47a9-49c0-89b1-437bd711374a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:10.257808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:10.282377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:10.282993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:10.283270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:10.284728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:10.285612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:10.285920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:10.287307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:10.307692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 208/417] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:10.315463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab046ad6-3b17-49f0-9a7b-6849a64806d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:10.316227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab046ad6-3b17-49f0-9a7b-6849a64806d4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:10.316511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab046ad6-3b17-49f0-9a7b-6849a64806d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:10.316808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab046ad6-3b17-49f0-9a7b-6849a64806d4 None admin] PATH_INFO: `/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:10.317957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab046ad6-3b17-49f0-9a7b-6849a64806d4 None admin] RBAC: Authorizing `identity:get_project(project_id=4b2aa0a8faf9471ebd699445da59c0dc)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:10.319405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab046ad6-3b17-49f0-9a7b-6849a64806d4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:10.321330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 210/418] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:10 2022] GET /identity/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:10.323446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:10.333495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab046ad6-3b17-49f0-9a7b-6849a64806d4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:10.463390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:10.516311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417cb63-ba46-4868-860e-eefb696815d2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:10.556718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab046ad6-3b17-49f0-9a7b-6849a64806d4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:10.584040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478a6d8-1877-45e1-923f-42b393bc4eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:10.584688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478a6d8-1877-45e1-923f-42b393bc4eff None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:10.585929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478a6d8-1877-45e1-923f-42b393bc4eff None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:10.586198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478a6d8-1877-45e1-923f-42b393bc4eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:10.630163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053992a7-c27c-442e-a030-7cb1e46d3558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:10.630852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053992a7-c27c-442e-a030-7cb1e46d3558 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:10.631141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053992a7-c27c-442e-a030-7cb1e46d3558 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:10.631421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053992a7-c27c-442e-a030-7cb1e46d3558 None admin] PATH_INFO: `/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc/groups/65a7482163dd46d9914ae19c6b323846/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:10.633391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053992a7-c27c-442e-a030-7cb1e46d3558 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b2aa0a8faf9471ebd699445da59c0dc, group_id=65a7482163dd46d9914ae19c6b323846, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:10.634874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053992a7-c27c-442e-a030-7cb1e46d3558 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:10.665247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 211/419] 10.222.0.95 () {58 vars in 1442 bytes} [Wed Aug 31 19:04:10 2022] PUT /identity/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc/groups/65a7482163dd46d9914ae19c6b323846/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:10.684085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478a6d8-1877-45e1-923f-42b393bc4eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:10.687428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478a6d8-1877-45e1-923f-42b393bc4eff None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:10.716962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddc134-5cae-49dc-826b-eb5157678f43 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:10.717136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddc134-5cae-49dc-826b-eb5157678f43 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:10.717413 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddc134-5cae-49dc-826b-eb5157678f43 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:10.722988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 212/420] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:10.724950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eeda9f-1ee2-4ad1-bf14-fe627cf743eb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:10.725312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eeda9f-1ee2-4ad1-bf14-fe627cf743eb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:10.725662 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eeda9f-1ee2-4ad1-bf14-fe627cf743eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:10.746175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a3eeda9f-1ee2-4ad1-bf14-fe627cf743eb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:10.747572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a3eeda9f-1ee2-4ad1-bf14-fe627cf743eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:10.797873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:10.808086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:10.841712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 209/421] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:10.847522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478a6d8-1877-45e1-923f-42b393bc4eff None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:10.886698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 213/422] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:10.970236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478a6d8-1877-45e1-923f-42b393bc4eff None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:11.016665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c86198-711a-4d39-9fd1-09575c1b7b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:11.017306 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86198-711a-4d39-9fd1-09575c1b7b31 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:11.017588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86198-711a-4d39-9fd1-09575c1b7b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:11.017862 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86198-711a-4d39-9fd1-09575c1b7b31 None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:11.019049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c86198-711a-4d39-9fd1-09575c1b7b31 None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:11.020561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c86198-711a-4d39-9fd1-09575c1b7b31 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:11.022528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 210/423] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:10 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:11.028531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eeda9f-1ee2-4ad1-bf14-fe627cf743eb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:11.175787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eeda9f-1ee2-4ad1-bf14-fe627cf743eb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:11.206144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c86198-711a-4d39-9fd1-09575c1b7b31 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:11.229365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b74ff-5038-4e33-92a4-95145d05312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:11.229927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b74ff-5038-4e33-92a4-95145d05312d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:11.230209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b74ff-5038-4e33-92a4-95145d05312d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:11.230479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b74ff-5038-4e33-92a4-95145d05312d None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:11.232646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b74ff-5038-4e33-92a4-95145d05312d None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:11.234159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b74ff-5038-4e33-92a4-95145d05312d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:11.236055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 214/424] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:11 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:11.241740 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b74ff-5038-4e33-92a4-95145d05312d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:11.256382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:11.300257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:11.321803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c86198-711a-4d39-9fd1-09575c1b7b31 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:11.377899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91804c-cdcf-4ec0-b5ea-35731ed88920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:11.378541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91804c-cdcf-4ec0-b5ea-35731ed88920 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:11.378816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91804c-cdcf-4ec0-b5ea-35731ed88920 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:11.379086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91804c-cdcf-4ec0-b5ea-35731ed88920 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:11.399140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91804c-cdcf-4ec0-b5ea-35731ed88920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-cf91804c-cdcf-4ec0-b5ea-35731ed88920 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:11.405611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:11.406550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:11.407880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 211/425] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:11.412616 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91804c-cdcf-4ec0-b5ea-35731ed88920 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:11.419936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b74ff-5038-4e33-92a4-95145d05312d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:11.494613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ba69b-32c3-41b3-90ed-5bcc6a112789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:11.495137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ba69b-32c3-41b3-90ed-5bcc6a112789 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:11.495409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ba69b-32c3-41b3-90ed-5bcc6a112789 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:11.495676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ba69b-32c3-41b3-90ed-5bcc6a112789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:11.496993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ba69b-32c3-41b3-90ed-5bcc6a112789 None admin] RBAC: Adding query filter params (group.id=ca9bd4608a6f4b86834b5b7f0d766ef7, role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:11.497469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ba69b-32c3-41b3-90ed-5bcc6a112789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:11.498914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ba69b-32c3-41b3-90ed-5bcc6a112789 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:11.510797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 215/426] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:11 2022] GET /identity/v3/role_assignments?group.id=ca9bd4608a6f4b86834b5b7f0d766ef7&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 611 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:11.535595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:11.540139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91804c-cdcf-4ec0-b5ea-35731ed88920 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:11.572861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dcbcb-7841-4e9c-96dd-57943bd04d21 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:11.573191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dcbcb-7841-4e9c-96dd-57943bd04d21 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:11.573397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dcbcb-7841-4e9c-96dd-57943bd04d21 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:11.575870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 216/427] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:11.581518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0b18-aaa5-4292-9f41-7c47f6f90787 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:11.581817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0b18-aaa5-4292-9f41-7c47f6f90787 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:11.582044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0b18-aaa5-4292-9f41-7c47f6f90787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:11.602492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-572b0b18-aaa5-4292-9f41-7c47f6f90787 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:11.603717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-572b0b18-aaa5-4292-9f41-7c47f6f90787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:11.615735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:11.616364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:11.616672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:11.616951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:11.617631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:11.617934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:11.619341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:11.642058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 212/428] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:04:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:11.648000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:11.680369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 217/429] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:11.795066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769e39-6f20-410f-a7de-121fbfd78d17 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:11.835197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7039b1-889c-492e-b8f0-231008427d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:11.836171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7039b1-889c-492e-b8f0-231008427d36 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:11.836711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7039b1-889c-492e-b8f0-231008427d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:11.837199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7039b1-889c-492e-b8f0-231008427d36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:11.869941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:11.913617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7039b1-889c-492e-b8f0-231008427d36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:11.915096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7039b1-889c-492e-b8f0-231008427d36 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:11.966668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b0b18-aaa5-4292-9f41-7c47f6f90787 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.002482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:12.055595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 213/430] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:04:12.062561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7039b1-889c-492e-b8f0-231008427d36 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.128948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b0b18-aaa5-4292-9f41-7c47f6f90787 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:12.144893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:12.176224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3038a4-9917-4075-871a-6f62fb324f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:12.177388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3038a4-9917-4075-871a-6f62fb324f33 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.177901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3038a4-9917-4075-871a-6f62fb324f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.178352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3038a4-9917-4075-871a-6f62fb324f33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.193230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3038a4-9917-4075-871a-6f62fb324f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-fd3038a4-9917-4075-871a-6f62fb324f33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:12.199287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:12.201034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:12.201844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 218/431] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:12.202498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47ca9-1dda-432e-b6de-0bcba42781d6 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.203068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47ca9-1dda-432e-b6de-0bcba42781d6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.203068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47ca9-1dda-432e-b6de-0bcba42781d6 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.204336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 219/432] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:12.204444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7039b1-889c-492e-b8f0-231008427d36 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:12.209539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eaf72-4d0c-4903-8abc-568afa587c2e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.209819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eaf72-4d0c-4903-8abc-568afa587c2e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.210084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eaf72-4d0c-4903-8abc-568afa587c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.230277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-491eaf72-4d0c-4903-8abc-568afa587c2e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:12.232035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-491eaf72-4d0c-4903-8abc-568afa587c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:12.253304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d943d-89e5-4c54-b47f-51089fea284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:12.253946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d943d-89e5-4c54-b47f-51089fea284b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.254207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d943d-89e5-4c54-b47f-51089fea284b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.254472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d943d-89e5-4c54-b47f-51089fea284b None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.255593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d943d-89e5-4c54-b47f-51089fea284b None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:12.257101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d943d-89e5-4c54-b47f-51089fea284b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:12.258879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 214/433] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:12.259475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d943d-89e5-4c54-b47f-51089fea284b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.318148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 220/434] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:12.318996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eaf72-4d0c-4903-8abc-568afa587c2e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.334920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:12.382021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d943d-89e5-4c54-b47f-51089fea284b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:12.413016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:12.424558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:12.425566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.425942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.426310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.427089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:12.427488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:12.428990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:12.443267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 215/435] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:12.444253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.446266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eaf72-4d0c-4903-8abc-568afa587c2e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:12.494670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a922a2-d37d-4a50-8825-ee17c9bb7bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:12.495557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a922a2-d37d-4a50-8825-ee17c9bb7bac None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.495840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a922a2-d37d-4a50-8825-ee17c9bb7bac None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.496111 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a922a2-d37d-4a50-8825-ee17c9bb7bac None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.497920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a922a2-d37d-4a50-8825-ee17c9bb7bac None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:12.499921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a922a2-d37d-4a50-8825-ee17c9bb7bac None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:12.501911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 221/436] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:12.507853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a922a2-d37d-4a50-8825-ee17c9bb7bac None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.565135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9414f8-7a59-471c-848c-c833ad16c286 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:12.609537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ad309-f320-4a41-b35f-40ec968fc8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:12.610057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ad309-f320-4a41-b35f-40ec968fc8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.610332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ad309-f320-4a41-b35f-40ec968fc8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.610596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ad309-f320-4a41-b35f-40ec968fc8e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.622893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:12.644773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a922a2-d37d-4a50-8825-ee17c9bb7bac None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:12.670933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:12.682610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ad309-f320-4a41-b35f-40ec968fc8e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:12.684303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ad309-f320-4a41-b35f-40ec968fc8e9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:12.685217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:12.685839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.686125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.686393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.687123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] RBAC: Adding query filter params (group.id=ca9bd4608a6f4b86834b5b7f0d766ef7, role.id=c2bcd89a362f4fb3a73e0f8ada320802, scope.project.id=362cde2d45474fbcadbd8b85adbbac97) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:12.687431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:12.688982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:12.695213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 222/437] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/role_assignments?group.id=ca9bd4608a6f4b86834b5b7f0d766ef7&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=c2bcd89a362f4fb3a73e0f8ada320802 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:12.695843 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.832945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 216/438] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:12.833936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ad309-f320-4a41-b35f-40ec968fc8e9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.857021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866eeec-fa29-4c8f-8dc3-d4060abbc5bc None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:12.896698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d3b5e-2e06-478f-9718-e1ee930b4bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:12.897255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d3b5e-2e06-478f-9718-e1ee930b4bcf None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:12.897533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d3b5e-2e06-478f-9718-e1ee930b4bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:12.897802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d3b5e-2e06-478f-9718-e1ee930b4bcf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:12.913717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d3b5e-2e06-478f-9718-e1ee930b4bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-8d9d3b5e-2e06-478f-9718-e1ee930b4bcf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:12.921436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:12.922388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:12.923215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 223/439] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:12.930371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d3b5e-2e06-478f-9718-e1ee930b4bcf None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:12.940360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:12.955813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:12.967177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ad309-f320-4a41-b35f-40ec968fc8e9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.004670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e27f75-e32d-4438-af33-f229a5a3a6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.005298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27f75-e32d-4438-af33-f229a5a3a6ac None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.005574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27f75-e32d-4438-af33-f229a5a3a6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.005843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e27f75-e32d-4438-af33-f229a5a3a6ac None admin] PATH_INFO: `/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.006946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e27f75-e32d-4438-af33-f229a5a3a6ac None admin] RBAC: Authorizing `identity:get_project(project_id=8f379c8467d44bcbbf9cbc6aac3a3231)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.008388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e27f75-e32d-4438-af33-f229a5a3a6ac None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.010263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 217/440] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/projects/8f379c8467d44bcbbf9cbc6aac3a3231 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:13.016023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e27f75-e32d-4438-af33-f229a5a3a6ac None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.047556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d3b5e-2e06-478f-9718-e1ee930b4bcf None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.083443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec3b92-b790-464b-a8fd-5d98725236c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec3b92-b790-464b-a8fd-5d98725236c4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.084339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec3b92-b790-464b-a8fd-5d98725236c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.084641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec3b92-b790-464b-a8fd-5d98725236c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.085311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec3b92-b790-464b-a8fd-5d98725236c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:13.085615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec3b92-b790-464b-a8fd-5d98725236c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.087015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec3b92-b790-464b-a8fd-5d98725236c4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.099038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 224/441] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:13.099038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68a83-6096-4912-b70c-8d5c70cd4f62 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.099038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68a83-6096-4912-b70c-8d5c70cd4f62 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.099038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68a83-6096-4912-b70c-8d5c70cd4f62 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.099728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 225/442] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:13.103203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e68a83-6096-4912-b70c-8d5c70cd4f62 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.133319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e27f75-e32d-4438-af33-f229a5a3a6ac None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.168204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de7af1-d8bf-4f2d-ae48-be65f7068208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.168714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7af1-d8bf-4f2d-ae48-be65f7068208 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.169509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7af1-d8bf-4f2d-ae48-be65f7068208 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.169775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7af1-d8bf-4f2d-ae48-be65f7068208 None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.170905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de7af1-d8bf-4f2d-ae48-be65f7068208 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.172327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de7af1-d8bf-4f2d-ae48-be65f7068208 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.173909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 218/443] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:13.175262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36cc9-cda7-4d3e-8f13-36c892df18ee None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.175535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36cc9-cda7-4d3e-8f13-36c892df18ee None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.175798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36cc9-cda7-4d3e-8f13-36c892df18ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.178560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:13.193397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e6a36cc9-cda7-4d3e-8f13-36c892df18ee None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:13.194589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e6a36cc9-cda7-4d3e-8f13-36c892df18ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:13.214496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:13.219469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e68a83-6096-4912-b70c-8d5c70cd4f62 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.257004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d84e63-1a11-4a69-a03d-94e0d3efba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.257640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84e63-1a11-4a69-a03d-94e0d3efba74 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.257920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84e63-1a11-4a69-a03d-94e0d3efba74 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.258185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84e63-1a11-4a69-a03d-94e0d3efba74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.261580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 219/444] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:13.262437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a36cc9-cda7-4d3e-8f13-36c892df18ee None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.336878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d84e63-1a11-4a69-a03d-94e0d3efba74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.338461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d84e63-1a11-4a69-a03d-94e0d3efba74 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.367286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a36cc9-cda7-4d3e-8f13-36c892df18ee None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.405241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.405806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.406092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.406359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.407102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] RBAC: Adding query filter params (group.id=ca9bd4608a6f4b86834b5b7f0d766ef7, role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:13.407412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.408884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.416169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:13.417401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 220/445] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/role_assignments?group.id=ca9bd4608a6f4b86834b5b7f0d766ef7&scope.project.id=8f379c8467d44bcbbf9cbc6aac3a3231&role.id=79dbf286cf884619af62f33ea75c52bf => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:13.466905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 226/446] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:13.472931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.499177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:13.536186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d84e63-1a11-4a69-a03d-94e0d3efba74 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.596681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1786a-42b6-4be6-ab4c-a54b86b04fec None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.633432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e3c6b-6974-45fa-8261-ef74abfca394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.634059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e3c6b-6974-45fa-8261-ef74abfca394 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.634334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e3c6b-6974-45fa-8261-ef74abfca394 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.634602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e3c6b-6974-45fa-8261-ef74abfca394 None admin] PATH_INFO: `/v3/projects/362cde2d45474fbcadbd8b85adbbac97` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.635717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e3c6b-6974-45fa-8261-ef74abfca394 None admin] RBAC: Authorizing `identity:get_project(project_id=362cde2d45474fbcadbd8b85adbbac97)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.637804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e3c6b-6974-45fa-8261-ef74abfca394 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.639616 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 221/447] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/projects/362cde2d45474fbcadbd8b85adbbac97 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:13.645029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e3c6b-6974-45fa-8261-ef74abfca394 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.662973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d84e63-1a11-4a69-a03d-94e0d3efba74 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.706710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86953c12-d425-4f6b-85a2-14e758639d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.707511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86953c12-d425-4f6b-85a2-14e758639d76 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.707850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86953c12-d425-4f6b-85a2-14e758639d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.708184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86953c12-d425-4f6b-85a2-14e758639d76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.721128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86953c12-d425-4f6b-85a2-14e758639d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-86953c12-d425-4f6b-85a2-14e758639d76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:13.726317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:13.727227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:13.728035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 227/448] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:13.735059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86953c12-d425-4f6b-85a2-14e758639d76 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.756782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:13.769901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e3c6b-6974-45fa-8261-ef74abfca394 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.771942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:13.814085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c967a-4c3d-4e80-839d-9d23b9996025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.814719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c967a-4c3d-4e80-839d-9d23b9996025 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.815001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c967a-4c3d-4e80-839d-9d23b9996025 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.815273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c967a-4c3d-4e80-839d-9d23b9996025 None admin] PATH_INFO: `/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.816381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c967a-4c3d-4e80-839d-9d23b9996025 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9bd4608a6f4b86834b5b7f0d766ef7)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.817875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c967a-4c3d-4e80-839d-9d23b9996025 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.819474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 222/449] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/groups/ca9bd4608a6f4b86834b5b7f0d766ef7 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:13.826067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c967a-4c3d-4e80-839d-9d23b9996025 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.861029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86953c12-d425-4f6b-85a2-14e758639d76 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.905294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.905894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.906169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.906431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.907093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:13.907389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.910004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.918023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 228/450] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:13.925092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:13.946187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c967a-4c3d-4e80-839d-9d23b9996025 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:13.980121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:13.981553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.982230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.982230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.982928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] RBAC: Adding query filter params (group.id=ca9bd4608a6f4b86834b5b7f0d766ef7, role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=362cde2d45474fbcadbd8b85adbbac97) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:13.983232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:13.984689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:13.990242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 223/451] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/role_assignments?group.id=ca9bd4608a6f4b86834b5b7f0d766ef7&scope.project.id=362cde2d45474fbcadbd8b85adbbac97&role.id=79dbf286cf884619af62f33ea75c52bf => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:14.021396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:14.032794 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdf6c1-051a-4ce1-ae65-533b321d34d3 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:14.069250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f0612-fa5b-42e3-b2bd-c57434855658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:14.069869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f0612-fa5b-42e3-b2bd-c57434855658 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:14.070146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f0612-fa5b-42e3-b2bd-c57434855658 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:14.070414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f0612-fa5b-42e3-b2bd-c57434855658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:14.150266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f0612-fa5b-42e3-b2bd-c57434855658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:14.151734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f0612-fa5b-42e3-b2bd-c57434855658 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:14.248086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:14.261268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 229/452] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:04:14.267263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:14.291122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:14.372565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6911e4-301a-4e80-af8e-c3d1de523f72 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:14.405958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d450c3c-7c74-4554-b0a8-412c3612b915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:14.406578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d450c3c-7c74-4554-b0a8-412c3612b915 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:14.406861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d450c3c-7c74-4554-b0a8-412c3612b915 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:14.407132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d450c3c-7c74-4554-b0a8-412c3612b915 None admin] PATH_INFO: `/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:14.408255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d450c3c-7c74-4554-b0a8-412c3612b915 None admin] RBAC: Authorizing `identity:get_project(project_id=4b2aa0a8faf9471ebd699445da59c0dc)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:14.409724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d450c3c-7c74-4554-b0a8-412c3612b915 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:14.411355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 224/453] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:04:14 2022] GET /identity/v3/projects/4b2aa0a8faf9471ebd699445da59c0dc => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:14.423280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f0612-fa5b-42e3-b2bd-c57434855658 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:14.519968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f0612-fa5b-42e3-b2bd-c57434855658 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:14.555370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91bd0-5263-4953-96c8-d0f97407d982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:14.555997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91bd0-5263-4953-96c8-d0f97407d982 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:14.556277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91bd0-5263-4953-96c8-d0f97407d982 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:14.556549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91bd0-5263-4953-96c8-d0f97407d982 None admin] PATH_INFO: `/v3/groups/65a7482163dd46d9914ae19c6b323846` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:14.557675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b91bd0-5263-4953-96c8-d0f97407d982 None admin] RBAC: Authorizing `identity:get_group(group_id=65a7482163dd46d9914ae19c6b323846)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:14.559338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b91bd0-5263-4953-96c8-d0f97407d982 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:14.561131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 230/454] 10.222.0.95 () {56 vars in 1180 bytes} [Wed Aug 31 19:04:14 2022] GET /identity/v3/groups/65a7482163dd46d9914ae19c6b323846 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:14.567656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d450c3c-7c74-4554-b0a8-412c3612b915 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:14.657397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:14.673619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d450c3c-7c74-4554-b0a8-412c3612b915 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:14.714317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fdc66-0b45-4fe6-b211-f7a4f9154eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:14.715656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fdc66-0b45-4fe6-b211-f7a4f9154eca None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:14.716157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fdc66-0b45-4fe6-b211-f7a4f9154eca None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:14.716669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fdc66-0b45-4fe6-b211-f7a4f9154eca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:14.717688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fdc66-0b45-4fe6-b211-f7a4f9154eca None admin] RBAC: Adding query filter params (group.id=65a7482163dd46d9914ae19c6b323846, role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=4b2aa0a8faf9471ebd699445da59c0dc) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:14.718002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fdc66-0b45-4fe6-b211-f7a4f9154eca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:14.719441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fdc66-0b45-4fe6-b211-f7a4f9154eca None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:14.726990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 225/455] 10.222.0.95 () {56 vars in 1376 bytes} [Wed Aug 31 19:04:14 2022] GET /identity/v3/role_assignments?group.id=65a7482163dd46d9914ae19c6b323846&scope.project.id=4b2aa0a8faf9471ebd699445da59c0dc&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:17.373437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7656-375c-4673-b0b7-54d73f3c4b9e None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.374463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7656-375c-4673-b0b7-54d73f3c4b9e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.375024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7656-375c-4673-b0b7-54d73f3c4b9e None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.376929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 231/456] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:17.382932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a7c8b-71c8-4609-982a-431124383284 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.383858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a7c8b-71c8-4609-982a-431124383284 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.384332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a7c8b-71c8-4609-982a-431124383284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.414397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e62a7c8b-71c8-4609-982a-431124383284 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:17.415946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e62a7c8b-71c8-4609-982a-431124383284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:17.429063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85d01c-e7f3-4b9c-87df-a4b2cdd9acea None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.429649 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85d01c-e7f3-4b9c-87df-a4b2cdd9acea None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.430128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85d01c-e7f3-4b9c-87df-a4b2cdd9acea None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.431858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 232/457] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:17.438264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c5c4d-08bb-4acf-b48d-448230c542c3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.438833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c5c4d-08bb-4acf-b48d-448230c542c3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.439318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c5c4d-08bb-4acf-b48d-448230c542c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.474759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-429c5c4d-08bb-4acf-b48d-448230c542c3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:17.476785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-429c5c4d-08bb-4acf-b48d-448230c542c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:17.535530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 226/458] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:17.581416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0a387-2bc5-41de-b993-c6d4d7cae653 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.583148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0a387-2bc5-41de-b993-c6d4d7cae653 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.584206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0a387-2bc5-41de-b993-c6d4d7cae653 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.587371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 227/459] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:17.595876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a0af5-7853-405b-9e00-54698c782df5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.596760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a0af5-7853-405b-9e00-54698c782df5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.597521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a0af5-7853-405b-9e00-54698c782df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.611444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:17.631815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a85a0af5-7853-405b-9e00-54698c782df5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:17.632979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a85a0af5-7853-405b-9e00-54698c782df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:17.640854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 233/460] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:17.738931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 228/461] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:17.845517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c5c4d-08bb-4acf-b48d-448230c542c3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:17.922289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42e165-345e-47d1-967a-751465899910 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.922996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42e165-345e-47d1-967a-751465899910 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.923473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42e165-345e-47d1-967a-751465899910 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.925644 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 229/462] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:17.938599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24552786-625a-4456-9c4c-f2164b2caf9b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.939327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24552786-625a-4456-9c4c-f2164b2caf9b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.939802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24552786-625a-4456-9c4c-f2164b2caf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.964083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c5c4d-08bb-4acf-b48d-448230c542c3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:17.984065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-24552786-625a-4456-9c4c-f2164b2caf9b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:17.985736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-24552786-625a-4456-9c4c-f2164b2caf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:18.002885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:18.010625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.011278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.011556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.012380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.035209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.037218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:18.054476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:18.055495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:18.056349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:18.056349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:18.056349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:18.058369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 234/463] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:18.071784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.107092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 230/464] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:18.108135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24552786-625a-4456-9c4c-f2164b2caf9b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.134271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:18.210510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9cd9c-eb49-4c15-bf80-3d87429d63cd None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:18.220488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24552786-625a-4456-9c4c-f2164b2caf9b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:18.252921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.253960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.254418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.254904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.255749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:18.256277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.257902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.258694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.259492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.260000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.260432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.272293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.274023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.276830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 235/465] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:18.277965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:18.284080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:18.297169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:18.298127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:18.298995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:18.298995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:18.298995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:18.301167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 231/466] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:18.307022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.398229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:18.402519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290693b-392e-4c38-9c3b-bbfcb3affe84 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:18.420026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620ff28-ded4-4185-8f4d-3fda9a485d0b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:18.442991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.443929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.444265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.444544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.457701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.458555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829ad28-9be9-44e3-8b8b-49c0627e2a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.459158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829ad28-9be9-44e3-8b8b-49c0627e2a86 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.459477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829ad28-9be9-44e3-8b8b-49c0627e2a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.459753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829ad28-9be9-44e3-8b8b-49c0627e2a86 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.460083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.460404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829ad28-9be9-44e3-8b8b-49c0627e2a86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.462150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829ad28-9be9-44e3-8b8b-49c0627e2a86 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:18.490918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:18.494303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:18.498593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:18.498593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:18.498593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:18.498873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 232/467] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:18 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:18.498873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829ad28-9be9-44e3-8b8b-49c0627e2a86 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.499084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 236/468] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:18.500305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.575577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:18.647437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829ad28-9be9-44e3-8b8b-49c0627e2a86 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:18.656017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9800e5-6541-45c8-bf1c-51761d65be2e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:18.690458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.691176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.691543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.691814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.692488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:18.692928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.694181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.698860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54057e58-80ca-4408-944b-671ad220a95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.699660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54057e58-80ca-4408-944b-671ad220a95a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.699961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54057e58-80ca-4408-944b-671ad220a95a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.700238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54057e58-80ca-4408-944b-671ad220a95a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.707281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 233/469] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:18.709486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.712994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54057e58-80ca-4408-944b-671ad220a95a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.714511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54057e58-80ca-4408-944b-671ad220a95a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-54057e58-80ca-4408-944b-671ad220a95a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:18.736121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:18.737160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:18.738019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:18.738019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:18.738019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:18.738019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 237/470] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:18.740008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bd92-a304-42d8-bafa-e085c348f35a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.740132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bd92-a304-42d8-bafa-e085c348f35a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.740532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95bd92-a304-42d8-bafa-e085c348f35a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.744642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 238/471] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:18.744642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95bd92-a304-42d8-bafa-e085c348f35a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.749502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:18.835232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556db46-4f6d-4c95-88f0-393a7f3f5f63 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:18.847636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:18.869966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95bd92-a304-42d8-bafa-e085c348f35a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:18.874927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.875536 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.875817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.876089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.876764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:18.877070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.878666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.892502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 234/472] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:18.893377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:18.915490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9afa9b-f02c-43c1-bff4-2a160221c438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:18.916101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9afa9b-f02c-43c1-bff4-2a160221c438 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.916382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9afa9b-f02c-43c1-bff4-2a160221c438 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.916716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9afa9b-f02c-43c1-bff4-2a160221c438 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.917341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9afa9b-f02c-43c1-bff4-2a160221c438 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:18.918848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9afa9b-f02c-43c1-bff4-2a160221c438 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:18.941430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 239/473] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:18 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:18.941430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e86189-913a-447d-addb-87f1d270a50f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:18.941430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e86189-913a-447d-addb-87f1d270a50f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:18.941430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e86189-913a-447d-addb-87f1d270a50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:18.971867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-62e86189-913a-447d-addb-87f1d270a50f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:18.973215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-62e86189-913a-447d-addb-87f1d270a50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:19.042083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc87d7-391e-4286-b2dc-982ce248d021 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:19.047028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:19.063661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 240/474] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:19.064608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e86189-913a-447d-addb-87f1d270a50f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:19.091459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:19.092747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.093681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.094305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.095233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:19.095942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:19.098224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:19.119745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 235/475] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:19.127642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:19.163160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:19.196531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e86189-913a-447d-addb-87f1d270a50f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:19.240057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563f5d-2ac8-44a3-930f-9778eab9cc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:19.240811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563f5d-2ac8-44a3-930f-9778eab9cc0b None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.241315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563f5d-2ac8-44a3-930f-9778eab9cc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.241713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563f5d-2ac8-44a3-930f-9778eab9cc0b None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.242431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519077f-726c-49bc-91da-1afc55c59c74 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:19.242773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7563f5d-2ac8-44a3-930f-9778eab9cc0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:19.245013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7563f5d-2ac8-44a3-930f-9778eab9cc0b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:19.273494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 241/476] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:19 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:19.283985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aff1aa-15b8-461c-aa99-2e389ea81674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:19.285952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aff1aa-15b8-461c-aa99-2e389ea81674 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.285952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aff1aa-15b8-461c-aa99-2e389ea81674 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.285952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aff1aa-15b8-461c-aa99-2e389ea81674 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.287370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aff1aa-15b8-461c-aa99-2e389ea81674 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:19.288903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aff1aa-15b8-461c-aa99-2e389ea81674 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:19.325248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 236/477] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:19 2022] POST /identity/v3/users => generated 249 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:19.431248 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563f5d-2ac8-44a3-930f-9778eab9cc0b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:19.497818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:19.560456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563f5d-2ac8-44a3-930f-9778eab9cc0b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:19.597884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:19.598501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.598780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.599048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.599737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:19.601198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:19.611021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.sql.core [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 19:04:19.611021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 19:04:19.611021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [parameters: {'id': '355a72f007904bb982b4a350cbcd26b2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 19:04:19.611021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81592) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application result.read() Aug 31 19:04:19.628628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:04:19.629561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application yield resource Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise self.value Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application session.commit() Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 19:04:19.630469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.session.flush() Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application compat.raise_( Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 19:04:19.631330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:04:19.633175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application result.read() Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:04:19.635153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [parameters: {'id': '355a72f007904bb982b4a350cbcd26b2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:19.638071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 19:04:19.639886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:19.641860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 242/478] 10.222.0.95 () {60 vars in 1131 bytes} [Wed Aug 31 19:04:19 2022] POST /identity/v3/roles => generated 144 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:19.641995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aff1aa-15b8-461c-aa99-2e389ea81674 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:19.732864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:19.753264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aff1aa-15b8-461c-aa99-2e389ea81674 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:19.800453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e7bff-9ef8-44c1-ab6a-b05277b79fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:19.801121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e7bff-9ef8-44c1-ab6a-b05277b79fbc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.801402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e7bff-9ef8-44c1-ab6a-b05277b79fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.801673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e7bff-9ef8-44c1-ab6a-b05277b79fbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.817863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e7bff-9ef8-44c1-ab6a-b05277b79fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-483e7bff-9ef8-44c1-ab6a-b05277b79fbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:19.823120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:19.823994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:19.825295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 237/479] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:04:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:19.835740 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:20.053517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eec9fe-c414-4ccf-9d5f-8e929a87e185 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:20.084980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:20.110375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493e46d-7060-4aed-9a66-df14b9ca037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:20.111005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493e46d-7060-4aed-9a66-df14b9ca037d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:20.111291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493e46d-7060-4aed-9a66-df14b9ca037d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:20.111568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493e46d-7060-4aed-9a66-df14b9ca037d None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:20.112244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493e46d-7060-4aed-9a66-df14b9ca037d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:20.112550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493e46d-7060-4aed-9a66-df14b9ca037d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:20.113985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493e46d-7060-4aed-9a66-df14b9ca037d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:20.146492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 243/480] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:20.872688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b042-7a91-4087-90aa-ef9b8a3c1a62 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:20.873261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b042-7a91-4087-90aa-ef9b8a3c1a62 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:20.873261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b042-7a91-4087-90aa-ef9b8a3c1a62 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:20.874676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 238/481] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:20.880617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a216a4-e0ec-45d5-aaad-ac5847642691 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:20.881119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a216a4-e0ec-45d5-aaad-ac5847642691 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:20.881119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a216a4-e0ec-45d5-aaad-ac5847642691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:20.906646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-11a216a4-e0ec-45d5-aaad-ac5847642691 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:20.907865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-11a216a4-e0ec-45d5-aaad-ac5847642691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:21.011029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 244/482] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:21.507567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387b042-7a91-4087-90aa-ef9b8a3c1a62 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:21.631213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387b042-7a91-4087-90aa-ef9b8a3c1a62 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:21.668682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b48976-28b0-4e6a-b99b-53c95bcd1f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:21.669327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b48976-28b0-4e6a-b99b-53c95bcd1f28 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:21.669607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b48976-28b0-4e6a-b99b-53c95bcd1f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:21.669877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b48976-28b0-4e6a-b99b-53c95bcd1f28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:21.681989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b48976-28b0-4e6a-b99b-53c95bcd1f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b2b48976-28b0-4e6a-b99b-53c95bcd1f28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:21.686877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:21.687825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:21.688646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 239/483] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:21.693943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a216a4-e0ec-45d5-aaad-ac5847642691 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:21.843682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a216a4-e0ec-45d5-aaad-ac5847642691 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:21.845451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:21.884830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4d552-e5d7-4819-8219-c686499a1755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:21.885596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4d552-e5d7-4819-8219-c686499a1755 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:21.885776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4d552-e5d7-4819-8219-c686499a1755 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:21.885961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4d552-e5d7-4819-8219-c686499a1755 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:21.886636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4d552-e5d7-4819-8219-c686499a1755 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:21.887681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4d552-e5d7-4819-8219-c686499a1755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:21.889109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4d552-e5d7-4819-8219-c686499a1755 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:21.906607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 245/484] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:21.911693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b48976-28b0-4e6a-b99b-53c95bcd1f28 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:21.933293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:22.004972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cde40-0a36-479b-b915-54694616b1a3 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.005390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cde40-0a36-479b-b915-54694616b1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.005620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cde40-0a36-479b-b915-54694616b1a3 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.007091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 246/485] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:22.014502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccfe2d-4782-4cfc-9c2f-f7802cbc2778 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.014783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccfe2d-4782-4cfc-9c2f-f7802cbc2778 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.015051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccfe2d-4782-4cfc-9c2f-f7802cbc2778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.045369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f6ccfe2d-4782-4cfc-9c2f-f7802cbc2778 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:22.046974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f6ccfe2d-4782-4cfc-9c2f-f7802cbc2778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:22.068919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b48976-28b0-4e6a-b99b-53c95bcd1f28 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:22.103146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69595e-0b92-49a3-ad65-d91c50c24cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:22.104344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69595e-0b92-49a3-ad65-d91c50c24cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.105115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69595e-0b92-49a3-ad65-d91c50c24cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.106675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69595e-0b92-49a3-ad65-d91c50c24cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.120531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69595e-0b92-49a3-ad65-d91c50c24cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:22.123424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69595e-0b92-49a3-ad65-d91c50c24cb4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-9c69595e-0b92-49a3-ad65-d91c50c24cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:22.145530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:22.146476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:22.147340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:22.147340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:22.147340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:22.149513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 240/486] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:22.150636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 247/487] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:22.151120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c416f-015c-45bf-8251-455853b0dda9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.151402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c416f-015c-45bf-8251-455853b0dda9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.151676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c416f-015c-45bf-8251-455853b0dda9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.160601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 241/488] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:22.160792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccfe2d-4782-4cfc-9c2f-f7802cbc2778 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:22.176245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c89cd9-831d-4f3d-88d8-4adb15ef7b33 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.176823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c89cd9-831d-4f3d-88d8-4adb15ef7b33 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.177308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c89cd9-831d-4f3d-88d8-4adb15ef7b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.207366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-87c89cd9-831d-4f3d-88d8-4adb15ef7b33 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:22.209424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-87c89cd9-831d-4f3d-88d8-4adb15ef7b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:22.282624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:22.290963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccfe2d-4782-4cfc-9c2f-f7802cbc2778 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:22.296820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:22.312779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 242/489] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:22.314341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951bda-fd26-498b-be71-e828cf105c1d None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.314801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951bda-fd26-498b-be71-e828cf105c1d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.315257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951bda-fd26-498b-be71-e828cf105c1d None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.320806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 243/490] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:22.322134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9a443-5999-4232-ae8e-12865731121c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.322694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9a443-5999-4232-ae8e-12865731121c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.323154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9a443-5999-4232-ae8e-12865731121c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.329055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:22.329669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.329951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.330223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.330869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:22.331257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:22.332599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:22.344303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-80a9a443-5999-4232-ae8e-12865731121c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:22.345766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 248/491] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:22.345997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-80a9a443-5999-4232-ae8e-12865731121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:22.358393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:22.478118 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 244/492] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:22.490949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9a443-5999-4232-ae8e-12865731121c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:22.491325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d306b5-2795-4ca2-8708-a877fb3d05c9 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:22.551274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:22.551906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.552181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.552511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.565777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:22.567996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:22.595991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:22.597000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:22.598080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:22.600021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 249/493] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:04:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:22.607806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:22.635447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:22.649135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9a443-5999-4232-ae8e-12865731121c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:22.673095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:22.696585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f8357-862d-476b-a67e-835e33830504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:22.697359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f8357-862d-476b-a67e-835e33830504 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.697766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f8357-862d-476b-a67e-835e33830504 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.698063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f8357-862d-476b-a67e-835e33830504 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.714587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f8357-862d-476b-a67e-835e33830504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-129f8357-862d-476b-a67e-835e33830504 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:22.720840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:22.721768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:22.722564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 245/494] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:22.723782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c861d2-6a4e-4e17-8acd-f996e819f909 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.723782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c861d2-6a4e-4e17-8acd-f996e819f909 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.724024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c861d2-6a4e-4e17-8acd-f996e819f909 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.726384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 246/495] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:22.729430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c861d2-6a4e-4e17-8acd-f996e819f909 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:22.754322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e2c8c-3b33-4345-a2c2-ad17670ee373 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:22.790653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:22.791258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.791644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.791967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.792726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:22.793075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:22.794726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:22.833955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 250/496] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:04:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:22.834863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:22.837249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c861d2-6a4e-4e17-8acd-f996e819f909 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:22.873017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948c7cc-8a0a-4a63-9c0c-854e62b1a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:22.873610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c7cc-8a0a-4a63-9c0c-854e62b1a17d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.873884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c7cc-8a0a-4a63-9c0c-854e62b1a17d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.874152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c7cc-8a0a-4a63-9c0c-854e62b1a17d None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.874807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948c7cc-8a0a-4a63-9c0c-854e62b1a17d None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:22.875429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948c7cc-8a0a-4a63-9c0c-854e62b1a17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:22.876511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948c7cc-8a0a-4a63-9c0c-854e62b1a17d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:22.884476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 247/497] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:22.889356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee117c-1921-40db-8b0e-e976f3948736 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.889946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee117c-1921-40db-8b0e-e976f3948736 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.890223 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee117c-1921-40db-8b0e-e976f3948736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.905571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:22.907438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6cee117c-1921-40db-8b0e-e976f3948736 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:22.908736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6cee117c-1921-40db-8b0e-e976f3948736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:22.937202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6aa20c-df76-43ce-80e0-e06740f8c109 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:22.941800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:22.974198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 248/498] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:22.974367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd74c2-2132-4385-b147-a2dabbfe5669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:22.974934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd74c2-2132-4385-b147-a2dabbfe5669 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.975032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee117c-1921-40db-8b0e-e976f3948736 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:22.975214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd74c2-2132-4385-b147-a2dabbfe5669 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.975494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd74c2-2132-4385-b147-a2dabbfe5669 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.995394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd74c2-2132-4385-b147-a2dabbfe5669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-00dd74c2-2132-4385-b147-a2dabbfe5669 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:23.000981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:23.001919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:23.002705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 251/499] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:23.003981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd74c2-2132-4385-b147-a2dabbfe5669 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.091609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee117c-1921-40db-8b0e-e976f3948736 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.117009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd74c2-2132-4385-b147-a2dabbfe5669 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.130655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb14e00-6cfc-46bd-b3dc-abf094684908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.131286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb14e00-6cfc-46bd-b3dc-abf094684908 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.131559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb14e00-6cfc-46bd-b3dc-abf094684908 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.131822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb14e00-6cfc-46bd-b3dc-abf094684908 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.143566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb14e00-6cfc-46bd-b3dc-abf094684908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1eb14e00-6cfc-46bd-b3dc-abf094684908 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:23.148266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:23.149193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 249/500] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:23.150881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb14e00-6cfc-46bd-b3dc-abf094684908 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.156257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.162436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b8c76-61ff-4f3c-b666-250cee444576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.163053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b8c76-61ff-4f3c-b666-250cee444576 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.163332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b8c76-61ff-4f3c-b666-250cee444576 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.163605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b8c76-61ff-4f3c-b666-250cee444576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.187248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.247584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b8c76-61ff-4f3c-b666-250cee444576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.249119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b8c76-61ff-4f3c-b666-250cee444576 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.260119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb14e00-6cfc-46bd-b3dc-abf094684908 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.294153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.294790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.295069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.295343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.305967 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.307429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:23.322850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:23.323907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:23.324859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:23.324859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.324859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:23.324859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 250/501] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:23.325287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.354281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 252/502] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:23.355128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b8c76-61ff-4f3c-b666-250cee444576 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.386120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.410510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.430044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760cb7c-9bbf-4243-b11c-4fba5fdd5363 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.456243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b8c76-61ff-4f3c-b666-250cee444576 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.463605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.464211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.464492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.465363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.466380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:23.466693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.468067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.476937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 251/503] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:23.477748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.490403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.491010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.491285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.491552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.492211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:23.492504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.493913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.502264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 253/504] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:23.503050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.579004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd60a0-b58f-42ee-b402-37b563c2083e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.601759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.609908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194684e9-f613-4b4d-82d3-3b18510e7410 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.612763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.613386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.613662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.613930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.625313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.626768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.629263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:23.642972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:23.644123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:23.645158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:23.645158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.645158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:23.645158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 252/505] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:23.645158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.647176 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.647960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.648231 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.649344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.649999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:23.650303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.651711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.663671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 254/506] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:23.664481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.749614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9690a-aaeb-4075-af68-1fe35b15888b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.769891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca2e52-b332-4e6d-94ae-ad09a2bd25d7 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.782709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.783329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.783606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.783872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.795387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.796867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.806002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.806627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.806906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.807179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:23.811677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:23.812816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:23.813727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:23.813727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.813727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:23.813727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 253/507] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:23.813992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.821117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.822580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:23.838148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:23.839272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:23.840147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:23.840147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.840147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:23.840147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 255/508] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:23.840147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.850654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.867043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.917734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d2e25-52ac-4f0d-b829-60628397f12a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.941268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02c7b8-a8cd-4f55-867d-cf359582fa8c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.952555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.953215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.953508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.953764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.964278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.965807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.974962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.975625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.975899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.976180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.976924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:23.977251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.978769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:23.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:23.982285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:23.983274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:23.983274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:23.983274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:23.983274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 254/509] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:23.983491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.992991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 256/510] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:23.993844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.059129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:24.086514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:24.089559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af024076-d142-4cf8-b2a4-9d86fab91095 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.096240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e57d0-2727-48cc-b767-dc694b4e7261 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.127688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.128325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.128632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.128905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.131429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.132034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.132310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.132594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.133253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:24.133570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.134993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.136784 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.140654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.147188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 257/511] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:24.148020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:24.153926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:24.154987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 255/512] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:24.155885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.257905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5193ef-c6ef-4a4c-a033-c7133c4ddfdd None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.266863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cac789-67b2-492c-a6fe-8af2a982e122 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.284077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:24.295304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.295908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.296185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.296455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.297131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:24.297428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.298810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.303305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.303906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.304192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.304465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.305147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:24.305460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.306839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.311057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 258/513] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:24.312991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.322988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 256/514] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:24.322988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.343698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:24.413844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b1651-af38-42d9-b4ef-4cb4d1769896 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.423848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350974b-8ed1-4ae6-bdbf-74dca9e865b1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.451238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a058d36-129a-462c-8aa1-7ee0484d3b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.451838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a058d36-129a-462c-8aa1-7ee0484d3b8d None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.452116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a058d36-129a-462c-8aa1-7ee0484d3b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.452388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a058d36-129a-462c-8aa1-7ee0484d3b8d None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.453062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a058d36-129a-462c-8aa1-7ee0484d3b8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.454538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a058d36-129a-462c-8aa1-7ee0484d3b8d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.457859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.458493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.458748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.459015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.465499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.466956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.474958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 259/515] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:24 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:24.475765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a058d36-129a-462c-8aa1-7ee0484d3b8d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:24.479037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:24.480113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 257/516] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:24.481120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.543120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:24.584250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:24.609408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900deb-6ca0-44a5-85dc-0915615feb89 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.617741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a058d36-129a-462c-8aa1-7ee0484d3b8d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.648323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.649066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.649347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.649615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.658541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.659964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.665884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.666508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.666788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.667355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.668770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:24.669159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.670601 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:24.673015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:24.674072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:04:24.675417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:24.676106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 258/517] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:24.680648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.692637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 260/518] 10.222.0.95 () {56 vars in 1137 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:24.693412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.788055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309245c-e0a1-4877-a8b5-b2511bcf8195 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.805529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:24.808445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378a825-b541-4368-b31a-9ddbf6152d58 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.830664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.831378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.831719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.832057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.839812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.841559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.844815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:24.845860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:24.846530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:24.846927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:24.847208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:24.847911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:24.848219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:24.849862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:24.855300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:24.857866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:04:24.859413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:24.862617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 259/519] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:24.863479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.877308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 261/520] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:24.877308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:24.977838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c29d79-d6df-48d5-8e70-de39df90a860 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:24.982636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688c78f-2c02-4fc4-b353-58cf0c3d50ab None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.017625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796726c-cef9-4fcc-a24c-55f3c1a71458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.018153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796726c-cef9-4fcc-a24c-55f3c1a71458 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.018433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796726c-cef9-4fcc-a24c-55f3c1a71458 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.018672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.018853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796726c-cef9-4fcc-a24c-55f3c1a71458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.019237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.019513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.019782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.021098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:25.021393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.022843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.052807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.066194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 260/521] 10.222.0.95 () {56 vars in 1137 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:25.067010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.098813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796726c-cef9-4fcc-a24c-55f3c1a71458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.100326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796726c-cef9-4fcc-a24c-55f3c1a71458 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.132960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.184201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c9d71-2a09-4f70-8cc8-9439333d24d8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.213215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 262/522] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:25.214274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796726c-cef9-4fcc-a24c-55f3c1a71458 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.232546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.233187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.233471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.233743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.234591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:25.234897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.236473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.260982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 261/523] 10.222.0.95 () {56 vars in 1139 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:25.260982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.282584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.330479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796726c-cef9-4fcc-a24c-55f3c1a71458 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.365373 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.365729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cf473-2689-4e40-9846-ed573d563be7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.365980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.366264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.366532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.367254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=aaab594854824542bd85426b0f16cbf6) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:25.367559 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.369029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.384600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 263/524] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/role_assignments?user.id=aaab594854824542bd85426b0f16cbf6&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:25.384600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.395178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.421121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17526d0f-8cb2-45c0-9802-c472cbec4b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.421892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17526d0f-8cb2-45c0-9802-c472cbec4b1f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.422276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17526d0f-8cb2-45c0-9802-c472cbec4b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.422712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17526d0f-8cb2-45c0-9802-c472cbec4b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.510923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d10a8d-62dd-4830-91fc-5172a30acc06 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.520807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17526d0f-8cb2-45c0-9802-c472cbec4b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.523536 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17526d0f-8cb2-45c0-9802-c472cbec4b1f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.540321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.557722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.559526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.559911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.560596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.575717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.577935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:25.599633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:25.600725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:25.601615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:25.601615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:25.601615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:25.601615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 264/525] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:25.602545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.664936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 262/526] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:25.665855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17526d0f-8cb2-45c0-9802-c472cbec4b1f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.684837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.730671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564152-22e8-4a6a-a040-8c50e99eb2f1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.769529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438b4f3-033f-4506-a24d-b798997d9257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.770438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438b4f3-033f-4506-a24d-b798997d9257 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.770894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438b4f3-033f-4506-a24d-b798997d9257 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.771368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438b4f3-033f-4506-a24d-b798997d9257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.771654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17526d0f-8cb2-45c0-9802-c472cbec4b1f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.806988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.807699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.808051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.808393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.809687 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:25.810056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.811507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.823420 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 263/527] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:25.824775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:25.825111 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.851955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438b4f3-033f-4506-a24d-b798997d9257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.853455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438b4f3-033f-4506-a24d-b798997d9257 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:25.910188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:25.938149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdafa32-b8c6-49a8-98d2-8137d441e34b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:25.974709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:25.975475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:25.975761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:25.976031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:25.987447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:25.989122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:26.006076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:26.007295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:26.008137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:26.008137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:26.008137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:26.009198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 265/529] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:26.010241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 264/529] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:26.010989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438b4f3-033f-4506-a24d-b798997d9257 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.011227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.098632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.125196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438b4f3-033f-4506-a24d-b798997d9257 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.129043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55b003-c441-49c6-8f82-591b45e16f93 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.162946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.166623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.167631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.169485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.170101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.173549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.174182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.174462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.175045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.178942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.180845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.187271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.188825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:26.195009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:26.195844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:26.197092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:26.197092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:26.197092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:26.197092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:26.197092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:26.197092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:26.197092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:26.197092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:26.204702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:26.205632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:26.206475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:26.206475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:26.206475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:26.206475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 265/531] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:26.207311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 266/531] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:26.207936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.209223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.325399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c755cee-fb9a-4f2e-9351-177c9dfc20f2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.331335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed9999-40cd-4639-990d-7aaf73a00b45 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.362988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c90042-1768-44db-b601-49682bf98962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.364014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c90042-1768-44db-b601-49682bf98962 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.364494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c90042-1768-44db-b601-49682bf98962 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.365007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c90042-1768-44db-b601-49682bf98962 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.366028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c90042-1768-44db-b601-49682bf98962 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:26.366548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c90042-1768-44db-b601-49682bf98962 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.368408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c90042-1768-44db-b601-49682bf98962 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.369232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.369840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.370126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.370404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.371953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:26.372464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.373773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.374117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.393693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 267/533] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:26.393693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c90042-1768-44db-b601-49682bf98962 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.393918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 266/533] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:26.393918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.442845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.503938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c90042-1768-44db-b601-49682bf98962 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.510336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe2f84-019b-4477-a883-9b529b3a4233 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.542196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c573a-a662-4c44-989f-4c04356eeeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.543492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c573a-a662-4c44-989f-4c04356eeeec None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.544387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c573a-a662-4c44-989f-4c04356eeeec None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.544692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c573a-a662-4c44-989f-4c04356eeeec None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.545376 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c573a-a662-4c44-989f-4c04356eeeec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:26.545638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c573a-a662-4c44-989f-4c04356eeeec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.547013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c573a-a662-4c44-989f-4c04356eeeec None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.547274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.547877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.548159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.548441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.549195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=8b6c90a2b95c498494a73b43e7138905) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:26.549505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.551181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.556623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 267/534] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/role_assignments?user.id=8b6c90a2b95c498494a73b43e7138905&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:26.557406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.560774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 268/535] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:26.562222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc57e7-96a4-4475-be25-e2bbd3dba41b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.562665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc57e7-96a4-4475-be25-e2bbd3dba41b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.563076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc57e7-96a4-4475-be25-e2bbd3dba41b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.564710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 269/536] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:26.569865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18149e36-1aa7-4dd3-b7ce-7d6baa481dd8 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.570154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18149e36-1aa7-4dd3-b7ce-7d6baa481dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.570414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18149e36-1aa7-4dd3-b7ce-7d6baa481dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.602717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-18149e36-1aa7-4dd3-b7ce-7d6baa481dd8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:26.604278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-18149e36-1aa7-4dd3-b7ce-7d6baa481dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:26.667666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.686133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c8c99-4171-481a-9aca-6b3d575391a6 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.688705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.691542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 270/537] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:26.692797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18149e36-1aa7-4dd3-b7ce-7d6baa481dd8 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.726567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.727233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.727604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.727883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.736225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.737722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:26.752174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:26.753203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:26.754316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:26.754316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:26.754316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:26.754316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:26.754316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:26.754316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:26.754316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:26.754316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:26.755324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 268/538] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:26.761056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.857397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18149e36-1aa7-4dd3-b7ce-7d6baa481dd8 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.932904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.935747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b26a19-1608-4482-a426-2f3815fb67c5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.937080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.937360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.937686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.951084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.952605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:26.965819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:26.966717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:26.967551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 271/539] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:26.972806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:27.022795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:27.023425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.023703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.023973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.024695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:27.025001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:27.026458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:27.028067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:27.048552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 269/540] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:27.049541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:27.081625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:27.106102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cee613-5c91-472b-8dcb-3bd48d665683 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:27.150337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:27.151616 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.152561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.153091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.154162 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:27.154719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:27.156511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:27.172309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 272/541] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:27.173491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:27.187118 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901e966-7367-408f-9f26-c10a7d432700 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:27.236619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cb574-fd9d-4524-bb50-69bf7e8a8fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:27.237413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cb574-fd9d-4524-bb50-69bf7e8a8fed None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.237790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cb574-fd9d-4524-bb50-69bf7e8a8fed None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.238260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cb574-fd9d-4524-bb50-69bf7e8a8fed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.250405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cb574-fd9d-4524-bb50-69bf7e8a8fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ef3cb574-fd9d-4524-bb50-69bf7e8a8fed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:27.259621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:27.260545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:27.261360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:27.268342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 270/542] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:27.268342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cb574-fd9d-4524-bb50-69bf7e8a8fed None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:27.312338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d32bd-4f39-4851-9ad1-d80486555a84 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:27.321356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:27.359417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:27.360048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.360324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.360620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.361683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=115298313c1243d282d09a8e8005002c) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:27.361995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:27.363439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:27.370232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 273/543] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:27 2022] GET /identity/v3/role_assignments?user.id=115298313c1243d282d09a8e8005002c&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:27.372342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:27.381712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:27.417674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cb574-fd9d-4524-bb50-69bf7e8a8fed None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:27.496822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becba4dc-4128-4b3c-aa03-d07fae7be815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:27.500161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becba4dc-4128-4b3c-aa03-d07fae7be815 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.501856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becba4dc-4128-4b3c-aa03-d07fae7be815 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.502461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becba4dc-4128-4b3c-aa03-d07fae7be815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.503583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becba4dc-4128-4b3c-aa03-d07fae7be815 None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=5f1d5c29df30465ead9834b63c3e93bb) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:27.504829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becba4dc-4128-4b3c-aa03-d07fae7be815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:27.506646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becba4dc-4128-4b3c-aa03-d07fae7be815 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:27.519228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 271/544] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:27 2022] GET /identity/v3/role_assignments?user.id=5f1d5c29df30465ead9834b63c3e93bb&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:27.540118 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6288-b50f-4d0d-8965-8879464f37c7 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.540661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6288-b50f-4d0d-8965-8879464f37c7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.541097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6288-b50f-4d0d-8965-8879464f37c7 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.544221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 272/545] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:27.550737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88f3d-481f-4736-af5a-063a3df64bec None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.551425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88f3d-481f-4736-af5a-063a3df64bec None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.551957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88f3d-481f-4736-af5a-063a3df64bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.575727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f150f-084a-45ca-8467-da206a524119 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:27.577294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-14d88f3d-481f-4736-af5a-063a3df64bec None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:27.579467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-14d88f3d-481f-4736-af5a-063a3df64bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:27.619412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:27.620014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.620349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.620655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.622513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:27.622723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:27.627441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:27.634095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 274/546] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:27.639934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:27.685043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:27.749760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 273/547] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:27.751400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50f647-e446-4f16-9f0e-626aa00686e9 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:27.789361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:27.790013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.790290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.790553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.804204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:27.805885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:27.822136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:27.823089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:27.824896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:27.824896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:27.824896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:27.824896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 275/548] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:27.829498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d88f3d-481f-4736-af5a-063a3df64bec None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:27.928209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:27.954867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d88f3d-481f-4736-af5a-063a3df64bec None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:27.997188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:27.997802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:27.998081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:27.998350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:27.998995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:27.999295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:28.000750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:28.016758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 274/549] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:28.019363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:28.116943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:28.126474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890ee23-7beb-409d-8f0a-5987c50dad51 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:28.161676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:28.162305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.162579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.162846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.170949 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:28.172434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:28.185488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:28.187944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:28.190163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 276/550] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:28.197113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:28.226777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:28.328517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9be421-e574-422f-9e60-04982f8df28d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:28.355262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed850-3ef0-44c8-ad31-59d48a32e29d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:28.362975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:28.363589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.363867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.364135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.364892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:28.365194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:28.366628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:28.386037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:28.386295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 275/551] 10.222.0.95 () {56 vars in 1135 bytes} [Wed Aug 31 19:04:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:28.393567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:28.417298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04e760-321d-4826-8500-360df71039e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:28.418479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e760-321d-4826-8500-360df71039e5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.418758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e760-321d-4826-8500-360df71039e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.419025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e760-321d-4826-8500-360df71039e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.436212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04e760-321d-4826-8500-360df71039e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-9c04e760-321d-4826-8500-360df71039e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:28.449151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:28.450301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:28.451110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 277/552] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:28.458098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04e760-321d-4826-8500-360df71039e5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:28.476972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:28.505581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0184a6-5b88-4cc3-8f99-c273e1a95a82 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:28.552043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d8331-543a-4bd1-acbf-bb3f8a6a41eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:28.552679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8331-543a-4bd1-acbf-bb3f8a6a41eb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.552964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8331-543a-4bd1-acbf-bb3f8a6a41eb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.553235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8331-543a-4bd1-acbf-bb3f8a6a41eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.622505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:28.627336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d8331-543a-4bd1-acbf-bb3f8a6a41eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:28.628845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d8331-543a-4bd1-acbf-bb3f8a6a41eb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:28.706627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04e760-321d-4826-8500-360df71039e5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:28.779631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:28.780239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.780516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.788889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.789607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:28.789915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:28.791309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:28.807333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 278/553] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:28.813475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 276/554] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:28.813751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:28.819413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d8331-543a-4bd1-acbf-bb3f8a6a41eb None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:28.963133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:28.964425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:28.972204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d8331-543a-4bd1-acbf-bb3f8a6a41eb None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.013140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fb8b4-1f4d-43d6-88c5-a7b3c9185e11 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.016123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eee7a-a257-4a2e-8438-21c10563bc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.016780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eee7a-a257-4a2e-8438-21c10563bc24 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.017056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eee7a-a257-4a2e-8438-21c10563bc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.017326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eee7a-a257-4a2e-8438-21c10563bc24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.029090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eee7a-a257-4a2e-8438-21c10563bc24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.030939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eee7a-a257-4a2e-8438-21c10563bc24 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6b7eee7a-a257-4a2e-8438-21c10563bc24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:29.049912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:29.051458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:29.053105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:29.053105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:29.053105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:29.053105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 277/555] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:29.053105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a45fc-0e91-47d2-b3d3-d42489282035 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.053105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a45fc-0e91-47d2-b3d3-d42489282035 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.053760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a45fc-0e91-47d2-b3d3-d42489282035 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.056662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 278/556] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:29.066825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a45fc-0e91-47d2-b3d3-d42489282035 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.071662 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6632d94-12ae-487c-a03b-1fe3443f2b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.072309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6632d94-12ae-487c-a03b-1fe3443f2b92 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.072608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6632d94-12ae-487c-a03b-1fe3443f2b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.072875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6632d94-12ae-487c-a03b-1fe3443f2b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.087433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6632d94-12ae-487c-a03b-1fe3443f2b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.089396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6632d94-12ae-487c-a03b-1fe3443f2b92 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f6632d94-12ae-487c-a03b-1fe3443f2b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:29.110092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:29.111530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:29.112431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:29.112431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:29.112431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:29.112431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 279/557] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:29.113127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0928-80dc-4cfb-87d5-deded5839d02 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.113303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0928-80dc-4cfb-87d5-deded5839d02 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.113531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b0928-80dc-4cfb-87d5-deded5839d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.138859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-aa3b0928-80dc-4cfb-87d5-deded5839d02 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:29.140085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-aa3b0928-80dc-4cfb-87d5-deded5839d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:29.208438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a45fc-0e91-47d2-b3d3-d42489282035 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.229257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 280/558] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:29.231677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b0928-80dc-4cfb-87d5-deded5839d02 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.283078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.284077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.284605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.285079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.285947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:29.286448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.289125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.315394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 279/559] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:29.320960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.344872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.349138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.451939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b0928-80dc-4cfb-87d5-deded5839d02 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.509279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979c407-eaec-41c7-9f6f-7a6881440811 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.535213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.535880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.536155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.536421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.537262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:29.537561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.540057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.569919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 281/560] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:29.573650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.579949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.580618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.580909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.581407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.598837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.600548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:29.618395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:29.619376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:29.620266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 280/561] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:29.621919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.725699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.741803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a566ebc-ed2e-438e-9296-0a77cbf0a8fc None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.753885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.770573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56210bd-edab-4d83-9d3f-b3ce957156c9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.788258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2d270-6056-4149-8fca-018157348891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.788929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2d270-6056-4149-8fca-018157348891 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.789209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2d270-6056-4149-8fca-018157348891 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.789476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2d270-6056-4149-8fca-018157348891 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.798154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2d270-6056-4149-8fca-018157348891 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.799895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2d270-6056-4149-8fca-018157348891 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.810030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a049a-43b8-4e05-b3d7-d32977f4fcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.810664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a049a-43b8-4e05-b3d7-d32977f4fcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.810945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a049a-43b8-4e05-b3d7-d32977f4fcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.811218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a049a-43b8-4e05-b3d7-d32977f4fcb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-cde2d270-6056-4149-8fca-018157348891 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:29.812361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:29.820734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 282/562] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:29.822688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2d270-6056-4149-8fca-018157348891 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.824542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a049a-43b8-4e05-b3d7-d32977f4fcb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d38a049a-43b8-4e05-b3d7-d32977f4fcb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:29.830329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:29.832444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 281/563] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:29.833393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2524f-d3fc-4269-a2b5-49152d7c4044 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.834234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2524f-d3fc-4269-a2b5-49152d7c4044 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.834234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2524f-d3fc-4269-a2b5-49152d7c4044 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.838679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 282/564] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:29.838679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ecc9-f726-4cc1-aac8-d2c16fab6c8d None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.838679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ecc9-f726-4cc1-aac8-d2c16fab6c8d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.838679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057ecc9-f726-4cc1-aac8-d2c16fab6c8d None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.838679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 283/565] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:29.838679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057ecc9-f726-4cc1-aac8-d2c16fab6c8d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.944961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2d270-6056-4149-8fca-018157348891 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.945670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057ecc9-f726-4cc1-aac8-d2c16fab6c8d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.972507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.979735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.980918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.981447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.981447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.981643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178afd83-ddec-48f2-87bd-771880baefb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.982137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:29.982234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178afd83-ddec-48f2-87bd-771880baefb0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.982442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.982534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178afd83-ddec-48f2-87bd-771880baefb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.982713 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178afd83-ddec-48f2-87bd-771880baefb0 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.983432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178afd83-ddec-48f2-87bd-771880baefb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:29.983739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178afd83-ddec-48f2-87bd-771880baefb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.983880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.985187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178afd83-ddec-48f2-87bd-771880baefb0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.995702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 283/566] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:29.997118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.003672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 284/567] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:30.004939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b0a4-ea0c-4d34-ba46-52b1de77ac23 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.005228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b0a4-ea0c-4d34-ba46-52b1de77ac23 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.005501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b0a4-ea0c-4d34-ba46-52b1de77ac23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.022746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-bce6b0a4-ea0c-4d34-ba46-52b1de77ac23 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:30.023882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bce6b0a4-ea0c-4d34-ba46-52b1de77ac23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:30.026854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.091006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 285/568] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:30.092245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe1665-4e5e-4629-9d09-80c095a0d3f7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.092531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe1665-4e5e-4629-9d09-80c095a0d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.092833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe1665-4e5e-4629-9d09-80c095a0d3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.099960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396b78-9c26-40f4-b612-b96a8b5dc6d3 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.119052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7bbe1665-4e5e-4629-9d09-80c095a0d3f7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:30.122911 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7bbe1665-4e5e-4629-9d09-80c095a0d3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:30.141413 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.142282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.142392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.142647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.143310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:30.143712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.145242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.158741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 284/569] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:30.158741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.208605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 286/570] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:30.208605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe1665-4e5e-4629-9d09-80c095a0d3f7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.260880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.289533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.296952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482b7e8-3f22-40dd-85a0-448b31f9256a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.332894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe1665-4e5e-4629-9d09-80c095a0d3f7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.335510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.336343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.336850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.337337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.338259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=bac9441fcf60412dbc33e0589a86c8ce) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:30.338796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.341498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.348153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 285/571] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/role_assignments?user.id=bac9441fcf60412dbc33e0589a86c8ce&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:30.349330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.388203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.388902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.389185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.389459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.401436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.403758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:30.421434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:30.422465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:30.423344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:30.423344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:30.423344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:30.427740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 287/572] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:30.428639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.489965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3527ad-62b3-43e2-9d7a-2e3506e6353f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.525975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.526743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.527029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.527298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.539831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.541268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.553399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.557116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67dcd-f619-4e3e-b4cc-899e4bc12347 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:30.560821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:30.561885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:30.563243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:30.563243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:30.563243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:30.563243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 286/573] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:30.563243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.576399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.598797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44337765-1fd6-4c2d-8936-2ece9348d4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.599517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337765-1fd6-4c2d-8936-2ece9348d4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.599801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337765-1fd6-4c2d-8936-2ece9348d4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.600072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337765-1fd6-4c2d-8936-2ece9348d4b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.612051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337765-1fd6-4c2d-8936-2ece9348d4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-44337765-1fd6-4c2d-8936-2ece9348d4b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:30.617449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:30.618556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:30.619537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 288/574] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:30.624055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44337765-1fd6-4c2d-8936-2ece9348d4b7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.681313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7005027-aa1e-4742-a128-7aae017cf575 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.719737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.720440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.720818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.721202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.721947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:30.722351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.723835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.740518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 287/575] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:30.741584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.744036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44337765-1fd6-4c2d-8936-2ece9348d4b7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.782969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ae25f-251d-41d8-9a74-139a7a869c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.783854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae25f-251d-41d8-9a74-139a7a869c14 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.784126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae25f-251d-41d8-9a74-139a7a869c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.784392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ae25f-251d-41d8-9a74-139a7a869c14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.795806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ae25f-251d-41d8-9a74-139a7a869c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f24ae25f-251d-41d8-9a74-139a7a869c14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:30.801260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:30.802428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 289/576] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:30.803484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ae25f-251d-41d8-9a74-139a7a869c14 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.821822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.829838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:30.866095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce83e51-8a88-4171-807a-dbd55e1ace73 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.915117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.915702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ae25f-251d-41d8-9a74-139a7a869c14 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:30.915890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.916233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.916585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.917382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:30.918786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.921375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.939380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 288/577] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:30.939380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:30.953885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:30.954492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:30.954766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:30.955033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:30.955692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:30.955994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:30.957413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:30.970961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 290/578] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:30.970961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.056172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.066369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6db4c-3348-4822-9d29-cac236aa830d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.088247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aaed5d-81c5-4078-a3ee-a005649092d1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.097085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.107871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.108755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.108901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.109070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.116403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.117931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.129743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.130453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:31.130747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:31.131778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:31.138577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:31.138577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:31.138577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:31.138577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:31.138577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:31.138577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:31.138577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:31.138577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:31.139112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.139112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.139273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 289/579] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:31.139367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:31.139451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.139539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.139539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.145698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 291/580] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:31.151268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.244261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df7512a-3fd0-4725-8526-a7beb09aa24b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.276885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66b6df-2ff8-4958-a866-a238d457ae93 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.283046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.283873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.284302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.284751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.293597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.295253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:31.307594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:31.308951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:04:31.310424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:31.313143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 290/581] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:31.313986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.318274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.319125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.319411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.319689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.333830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.335293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.337705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:31.353181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:31.354157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:31.355049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:31.355049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:31.355049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:31.358996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 292/582] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:31.359919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.364609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.428752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15021b-a57f-4775-9245-d9026ab08847 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.472323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.473112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.473314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.473580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.474270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:31.474571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.474910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75222cc-ccf7-463a-80ff-05a2e3279a55 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.475998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.489329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 291/583] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:31.489956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.527418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.528383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.528709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.528986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.539770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.541244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:31.555592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:31.556535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:31.557480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:31.557480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:31.557480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:31.557480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 293/584] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:31.557480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.603375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653dd54-568a-43c4-aa4d-6798cda9c7c6 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.611672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.612416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.641525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.642133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.642415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.642689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.643421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:31.643735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.645234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.663897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 292/585] 10.222.0.95 () {56 vars in 1137 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:31.664972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.678209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925db14-dcfc-4f9f-993f-4bd56c65b7e1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.717065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.717680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.718009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.718286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.719017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:31.719429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.720938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.739373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 294/586] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:31.739373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.778379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f4c53-cfdc-430b-aa47-8ec7457ff815 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.815928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0831-bfbd-4ecd-9b31-ab32e3f21ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.816612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0831-bfbd-4ecd-9b31-ab32e3f21ea5 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.816935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0831-bfbd-4ecd-9b31-ab32e3f21ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.817578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce0831-bfbd-4ecd-9b31-ab32e3f21ea5 None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/aaab594854824542bd85426b0f16cbf6/roles/c49d2fa980524a9f95d78b7fff997d1c` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.835065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce0831-bfbd-4ecd-9b31-ab32e3f21ea5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=aaab594854824542bd85426b0f16cbf6, role_id=c49d2fa980524a9f95d78b7fff997d1c)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.836528 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce0831-bfbd-4ecd-9b31-ab32e3f21ea5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.848959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4be83-6854-451f-9233-6eba205990bf None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:31.850816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 293/587] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:31 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/aaab594854824542bd85426b0f16cbf6/roles/c49d2fa980524a9f95d78b7fff997d1c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:31.851439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0831-bfbd-4ecd-9b31-ab32e3f21ea5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:31.872085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.873243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.924411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:31.925758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.926447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.926874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.928245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:31.928734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:31.930342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:31.950980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 295/588] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:31.952130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.006063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce0831-bfbd-4ecd-9b31-ab32e3f21ea5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.043849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.044490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.044802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.045074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.055777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.057304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.070486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1cf77-3373-4cad-83f7-c0d9944eb493 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:32.073177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:32.074952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:32.076178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:32.076178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:32.076178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:32.079762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 294/589] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:32.080799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.115107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.116062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.117250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.117631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.132501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.134622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:32.156320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:32.157383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:04:32.158344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:32.162121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 296/590] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:32.162979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.163399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:32.192945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:32.211270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df44ca1-12ae-464e-a2d1-62f9d102da26 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.257280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.258143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.258879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.259155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.270459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.272722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.279713 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017186e7-431c-4b9c-bfc9-77690345afc1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:32.288293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:32.289303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:32.290265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 295/591] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:32.290873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.331685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.332297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.332618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.332885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.333533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:32.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.335237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.348727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 297/592] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:32.349571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.405420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd561a-eae3-43fe-9415-9a5155f89b6e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.434220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:32.444400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3612ad-ec0f-4cd1-9ec1-329a09dbdcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.445052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3612ad-ec0f-4cd1-9ec1-329a09dbdcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.445336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3612ad-ec0f-4cd1-9ec1-329a09dbdcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.445611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3612ad-ec0f-4cd1-9ec1-329a09dbdcb4 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.446346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3612ad-ec0f-4cd1-9ec1-329a09dbdcb4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:32.446658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3612ad-ec0f-4cd1-9ec1-329a09dbdcb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.448176 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3612ad-ec0f-4cd1-9ec1-329a09dbdcb4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.454038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:32.471293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960742-6a70-403a-9efb-6dd0e17256bb None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.476164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 296/593] 10.222.0.95 () {56 vars in 1137 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:32.476164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd521727-7b28-44ee-b35a-e9dd3e37bc4e None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.476164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd521727-7b28-44ee-b35a-e9dd3e37bc4e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.476164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd521727-7b28-44ee-b35a-e9dd3e37bc4e None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.476164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 297/594] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:32.476546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd521727-7b28-44ee-b35a-e9dd3e37bc4e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.514775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.515557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.515936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.516337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.517157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:32.518031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.519048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.542043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 298/595] 10.222.0.95 () {56 vars in 1139 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:32.542043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.594483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd521727-7b28-44ee-b35a-e9dd3e37bc4e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.632159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42afbd-5643-4d2e-922a-76e68cf98dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.632859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42afbd-5643-4d2e-922a-76e68cf98dbd None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.633141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42afbd-5643-4d2e-922a-76e68cf98dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.633410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42afbd-5643-4d2e-922a-76e68cf98dbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.640909 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42afbd-5643-4d2e-922a-76e68cf98dbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.642470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42afbd-5643-4d2e-922a-76e68cf98dbd None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.646312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d15a8-e636-4549-bb9a-032ecfc13404 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-df42afbd-5643-4d2e-922a-76e68cf98dbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:32.655039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:32.656012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:32.657326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 298/596] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:32.657812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea641eb-fc00-43cd-acfc-3c41e4af58f6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.657812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea641eb-fc00-43cd-acfc-3c41e4af58f6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.657948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea641eb-fc00-43cd-acfc-3c41e4af58f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.678961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-eea641eb-fc00-43cd-acfc-3c41e4af58f6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:32.680667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-eea641eb-fc00-43cd-acfc-3c41e4af58f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:32.689026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.689666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.689950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.690228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.704638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.707064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.715067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:32.731877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:32.733112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:32.734133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:32.734133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:32.734133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:32.734133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 299/597] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:32.734452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.739545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:32.770633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 299/598] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:32.771290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea641eb-fc00-43cd-acfc-3c41e4af58f6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.874594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd27180-22ef-4b69-a713-eb25d5daad01 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.889510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea641eb-fc00-43cd-acfc-3c41e4af58f6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.910932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.911474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.911759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.912036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.924158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.925991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.933112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.934034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.934654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.935125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.936019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:32.936521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.938186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:32.941293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:32.942267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:32.943638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:32.943638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:32.943638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:32.946843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 300/599] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:32.947678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.955535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 300/600] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:32.956810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.996401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.017327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.076409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66bfae-afc4-4ce3-8f59-40ffd75e55ad None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.079770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1ba0b-5420-4ec0-a35d-8196c7b942bc None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.124860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.126638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.126920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.127190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.127361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.127844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:33.128144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.128243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.128479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.128858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.129556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.129727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:33.129871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.131353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.145492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 301/601] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:33.146232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.153846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 301/602] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:33.155454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.271677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.284221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0a1a8-6a09-4f97-83cb-8497f583bfe5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.291730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fdd3f-f519-43c8-86a4-3f86340f2e88 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.303917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.327024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f8cb7-30e8-4fea-a463-ea1cbe14949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.327816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f8cb7-30e8-4fea-a463-ea1cbe14949c None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.328107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f8cb7-30e8-4fea-a463-ea1cbe14949c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.328382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f8cb7-30e8-4fea-a463-ea1cbe14949c None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/8b6c90a2b95c498494a73b43e7138905/roles/c49d2fa980524a9f95d78b7fff997d1c` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.336258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c961e-d017-4605-b634-fd7799b30859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.337011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c961e-d017-4605-b634-fd7799b30859 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.337362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c961e-d017-4605-b634-fd7799b30859 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.337695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c961e-d017-4605-b634-fd7799b30859 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.348127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f8cb7-30e8-4fea-a463-ea1cbe14949c None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=8b6c90a2b95c498494a73b43e7138905, role_id=c49d2fa980524a9f95d78b7fff997d1c)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.349978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f8cb7-30e8-4fea-a463-ea1cbe14949c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.350111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c961e-d017-4605-b634-fd7799b30859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-798c961e-d017-4605-b634-fd7799b30859 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:33.354866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:33.356995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 302/603] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:33.358111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c961e-d017-4605-b634-fd7799b30859 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.370948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 302/604] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:33 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/8b6c90a2b95c498494a73b43e7138905/roles/c49d2fa980524a9f95d78b7fff997d1c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:33.372879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f8cb7-30e8-4fea-a463-ea1cbe14949c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.524673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c961e-d017-4605-b634-fd7799b30859 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.533462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f8cb7-30e8-4fea-a463-ea1cbe14949c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.571251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.571892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.572172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.572442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.580838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.582323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.584747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.585709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.586201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.586667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.595888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.597673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:33.600281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:33.601344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:33.602383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:33.602383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:33.602383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:33.602383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:33.602383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:33.602383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:33.602383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:33.602383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:33.607781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 303/605] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:33.608869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:33.617327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:33.618615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:33.619729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:33.619729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:33.619729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:33.619729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:33.619729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:33.619729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:33.619729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:33.619729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:33.622272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 303/606] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:33.628928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.634723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.648135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.786202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5572f0-cf4c-42d2-af9a-97a3654dfc6b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.812784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21411c8c-18bb-4163-bf58-aad9aa549cf1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:33.839587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.840382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.841239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.841611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.842453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:33.842845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.844450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.857177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 304/607] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:33.858493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.863938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:33.864548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:33.864851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:33.865117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:33.865816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:33.866116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:33.867566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:33.892807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 304/608] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:33.894055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:33.973500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.974629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:33.990949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471b137-656e-4d27-82bf-c14d60006a66 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:34.021107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52507e3-00e2-44ac-83fe-1e2485a5b631 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:34.045959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:34.046502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.046785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.047062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.047762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:34.048067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:34.049803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:34.064280 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 305/609] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:34.064518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa74de7-96a9-4e79-9d0a-27d88700b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:34.065116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa74de7-96a9-4e79-9d0a-27d88700b6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.065395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa74de7-96a9-4e79-9d0a-27d88700b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.065670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa74de7-96a9-4e79-9d0a-27d88700b6d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.066054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:34.081839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa74de7-96a9-4e79-9d0a-27d88700b6d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:34.083321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa74de7-96a9-4e79-9d0a-27d88700b6d8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-5aa74de7-96a9-4e79-9d0a-27d88700b6d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:34.099803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:34.100845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:34.101867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:34.101867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:34.101867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:34.106914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 305/610] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:34.108349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a472f0a-1d77-4576-8dde-ba6f85d08327 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.108656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a472f0a-1d77-4576-8dde-ba6f85d08327 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.108931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a472f0a-1d77-4576-8dde-ba6f85d08327 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.110266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 306/611] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:34.115151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a472f0a-1d77-4576-8dde-ba6f85d08327 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:34.226067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac618a36-f26a-4881-8618-5f1e4154fef3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:34.266318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a472f0a-1d77-4576-8dde-ba6f85d08327 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:34.287620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69e7e9-60d7-4d67-94fb-f21e573b4305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:34.288918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69e7e9-60d7-4d67-94fb-f21e573b4305 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.289351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69e7e9-60d7-4d67-94fb-f21e573b4305 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.289819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69e7e9-60d7-4d67-94fb-f21e573b4305 None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/5f1d5c29df30465ead9834b63c3e93bb/roles/c49d2fa980524a9f95d78b7fff997d1c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.290656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:34.308858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69e7e9-60d7-4d67-94fb-f21e573b4305 None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=5f1d5c29df30465ead9834b63c3e93bb, role_id=c49d2fa980524a9f95d78b7fff997d1c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:34.310244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69e7e9-60d7-4d67-94fb-f21e573b4305 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:34.316532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269e09e-1346-4a60-b3d8-95927d4b194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:34.316811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e09e-1346-4a60-b3d8-95927d4b194a None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.316970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e09e-1346-4a60-b3d8-95927d4b194a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.317164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e09e-1346-4a60-b3d8-95927d4b194a None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/115298313c1243d282d09a8e8005002c/roles/c49d2fa980524a9f95d78b7fff997d1c` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.323691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:34.344612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 306/612] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:34 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/5f1d5c29df30465ead9834b63c3e93bb/roles/c49d2fa980524a9f95d78b7fff997d1c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:34.346477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69e7e9-60d7-4d67-94fb-f21e573b4305 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:34.353234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269e09e-1346-4a60-b3d8-95927d4b194a None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=115298313c1243d282d09a8e8005002c, role_id=c49d2fa980524a9f95d78b7fff997d1c)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:34.355639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269e09e-1346-4a60-b3d8-95927d4b194a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:34.389228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 307/613] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:34 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/115298313c1243d282d09a8e8005002c/roles/c49d2fa980524a9f95d78b7fff997d1c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:34.391567 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4980-3f59-44a5-9bc5-d37f4f6cc79e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.391934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4980-3f59-44a5-9bc5-d37f4f6cc79e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.392222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a4980-3f59-44a5-9bc5-d37f4f6cc79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.423357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-ab2a4980-3f59-44a5-9bc5-d37f4f6cc79e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:34.425672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ab2a4980-3f59-44a5-9bc5-d37f4f6cc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:34.556236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 308/614] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:34.588859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69e7e9-60d7-4d67-94fb-f21e573b4305 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:34.626924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:34.627539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.627825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.628091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.628779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:34.629080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:34.630535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:34.648147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 307/615] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:34.656315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a4980-3f59-44a5-9bc5-d37f4f6cc79e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:34.831738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:34.842005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a4980-3f59-44a5-9bc5-d37f4f6cc79e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:34.901822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:34.902948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.903427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.903889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.917078 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:34.920520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:34.943904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:34.945855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:34.947022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 309/616] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:34.956697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.042118 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:35.098732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d4a8f-00f6-4500-b980-0cec4786cf59 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:35.149919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:35.150594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.150873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.151190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.151951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:35.152784 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:35.154307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:35.179935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 308/617] 10.222.0.95 () {56 vars in 1135 bytes} [Wed Aug 31 19:04:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:35.186130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.233700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.334929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d827c8-9225-4104-87c7-59cfc6eb137f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:35.409226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:35.411037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.411316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.411588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.426825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568e591-f2fc-4d36-9f4d-8649b2b4c546 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:35.439769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:35.441269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:35.460955 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:35.470568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:35.471558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:35.472538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:35.472538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:35.472538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:35.476621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 310/618] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:35.477857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.499141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:35.505107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee395f-242a-41f1-a9db-3d9f04b6d583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:35.508803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee395f-242a-41f1-a9db-3d9f04b6d583 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.508803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee395f-242a-41f1-a9db-3d9f04b6d583 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.508803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee395f-242a-41f1-a9db-3d9f04b6d583 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.532097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee395f-242a-41f1-a9db-3d9f04b6d583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-95ee395f-242a-41f1-a9db-3d9f04b6d583 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:35.543790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:35.546137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:35.548103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:35.556587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 309/619] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:35.564846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee395f-242a-41f1-a9db-3d9f04b6d583 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.641491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e000ec-25ab-4f5d-90d6-faf297879830 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:35.689367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e3d3e-f447-48c7-81ec-c192b8dbed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:35.690289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e3d3e-f447-48c7-81ec-c192b8dbed4e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.690697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e3d3e-f447-48c7-81ec-c192b8dbed4e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.691083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e3d3e-f447-48c7-81ec-c192b8dbed4e None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.692056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e3d3e-f447-48c7-81ec-c192b8dbed4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:35.692429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e3d3e-f447-48c7-81ec-c192b8dbed4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:35.694613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e3d3e-f447-48c7-81ec-c192b8dbed4e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:35.695852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee395f-242a-41f1-a9db-3d9f04b6d583 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:35.711135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 311/620] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:35.713208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990523c-bcd5-4c02-bdb6-ac614d35e363 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.713495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990523c-bcd5-4c02-bdb6-ac614d35e363 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.713769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990523c-bcd5-4c02-bdb6-ac614d35e363 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.716160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 312/621] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:35.716900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990523c-bcd5-4c02-bdb6-ac614d35e363 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.738552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42dafa0-d246-438f-996a-740224e269f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:35.739180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dafa0-d246-438f-996a-740224e269f7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.739462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dafa0-d246-438f-996a-740224e269f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.739742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dafa0-d246-438f-996a-740224e269f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.740419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dafa0-d246-438f-996a-740224e269f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:35.740765 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dafa0-d246-438f-996a-740224e269f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:35.742145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dafa0-d246-438f-996a-740224e269f7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:35.752413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 310/622] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:35.758641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2422-e384-47cd-a907-c9993ff838e8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.759026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2422-e384-47cd-a907-c9993ff838e8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.759308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e2422-e384-47cd-a907-c9993ff838e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.778894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-248e2422-e384-47cd-a907-c9993ff838e8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:35.780123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-248e2422-e384-47cd-a907-c9993ff838e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:35.797933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:35.819268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:35.838954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990523c-bcd5-4c02-bdb6-ac614d35e363 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:35.860604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 311/623] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:35.860604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e2422-e384-47cd-a907-c9993ff838e8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.904091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:35.905015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.905449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.905872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.920176 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:35.928595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:35.948289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:35.949886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:35.951323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:35.951323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:35.951323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:35.951323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:35.951323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:35.951323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:35.951323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:35.951323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:35.953375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 313/624] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:35.958732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.967937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e2422-e384-47cd-a907-c9993ff838e8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.004973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.005659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.005887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.012722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.017141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.018610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:36.036051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:36.037083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:36.037987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:36.037987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:36.037987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:36.037987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 312/625] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:36.043976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.071319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:36.125127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e62f59-39e8-4709-b6e7-d70ab6083b78 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.158302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:36.172067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.173152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.173435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.173711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.174437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:36.174992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.179385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.205591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 314/626] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:36.209657 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7044543-6a65-4958-a30a-ad48c24c2a1e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.210990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.278244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.279064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.279345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.279619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.280271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:36.280600 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.281976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.314028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 313/627] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:36.319423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.372512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7cb2d-33d1-4ef9-b016-28766dd2bb11 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.409970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed4e0f-6c44-4189-a36b-5b5dfec87a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.410641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4e0f-6c44-4189-a36b-5b5dfec87a16 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.410924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4e0f-6c44-4189-a36b-5b5dfec87a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.411201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4e0f-6c44-4189-a36b-5b5dfec87a16 None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/bac9441fcf60412dbc33e0589a86c8ce/roles/c49d2fa980524a9f95d78b7fff997d1c` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.426156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:36.427811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed4e0f-6c44-4189-a36b-5b5dfec87a16 None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=bac9441fcf60412dbc33e0589a86c8ce, role_id=c49d2fa980524a9f95d78b7fff997d1c)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.429794 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed4e0f-6c44-4189-a36b-5b5dfec87a16 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.444529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 315/628] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:36 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/bac9441fcf60412dbc33e0589a86c8ce/roles/c49d2fa980524a9f95d78b7fff997d1c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:36.445675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed4e0f-6c44-4189-a36b-5b5dfec87a16 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.451112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a74ea-4310-4099-8640-81c39e0a1048 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.515606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.516765 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.517280 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.517791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.537427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:36.537992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.540191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:36.567731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:36.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:36.569738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:36.572371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 314/629] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:04:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:36.587895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.622188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed4e0f-6c44-4189-a36b-5b5dfec87a16 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.663274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea22af1-7ec4-4db1-ba06-4752c5ed9351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.664266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea22af1-7ec4-4db1-ba06-4752c5ed9351 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.664784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea22af1-7ec4-4db1-ba06-4752c5ed9351 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.665291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea22af1-7ec4-4db1-ba06-4752c5ed9351 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.682058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea22af1-7ec4-4db1-ba06-4752c5ed9351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-4ea22af1-7ec4-4db1-ba06-4752c5ed9351 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:36.687588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:36.688516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:36.689684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 316/630] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:36.695008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deedac-9486-4e35-8b1e-a28409a4c32b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.695008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deedac-9486-4e35-8b1e-a28409a4c32b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.695008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deedac-9486-4e35-8b1e-a28409a4c32b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.695008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 317/631] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:36.697360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deedac-9486-4e35-8b1e-a28409a4c32b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.734667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b5277-c210-46bf-b59b-ecea1dcdd161 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.792210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e2a35-ab25-442c-ba29-a8cdc40dea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.794883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2a35-ab25-442c-ba29-a8cdc40dea72 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.795512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2a35-ab25-442c-ba29-a8cdc40dea72 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.796055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e2a35-ab25-442c-ba29-a8cdc40dea72 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.797261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e2a35-ab25-442c-ba29-a8cdc40dea72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:36.797723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e2a35-ab25-442c-ba29-a8cdc40dea72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.800264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e2a35-ab25-442c-ba29-a8cdc40dea72 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.817841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:36.828392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 315/632] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:04:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:36.830109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c00e-7af6-40c7-b57c-901a8e5d813e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.830561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c00e-7af6-40c7-b57c-901a8e5d813e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.830981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c00e-7af6-40c7-b57c-901a8e5d813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.844048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:36.852905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deedac-9486-4e35-8b1e-a28409a4c32b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:36.854127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a031c00e-7af6-40c7-b57c-901a8e5d813e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:36.856613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a031c00e-7af6-40c7-b57c-901a8e5d813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:36.905694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:36.906134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:36.906586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:36.906958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:36.907818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:36.908224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:36.909782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:36.925217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 318/633] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:36.926367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:36.941482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 316/634] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:36.943132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031c00e-7af6-40c7-b57c-901a8e5d813e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.050861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6ad50-1b9f-439b-bc9d-5f5f49a63cc1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:37.066055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031c00e-7af6-40c7-b57c-901a8e5d813e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:37.096552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4ed08-a5b6-451d-8153-0a5b818281d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:37.097110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4ed08-a5b6-451d-8153-0a5b818281d3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.097395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4ed08-a5b6-451d-8153-0a5b818281d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.097669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4ed08-a5b6-451d-8153-0a5b818281d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.103190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:37.110746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b0706-688c-4995-9c15-d8f424f53fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:37.111689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b0706-688c-4995-9c15-d8f424f53fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.112171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b0706-688c-4995-9c15-d8f424f53fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.112766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b0706-688c-4995-9c15-d8f424f53fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.126182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b0706-688c-4995-9c15-d8f424f53fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:37.127954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b0706-688c-4995-9c15-d8f424f53fb5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:37.133556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3e0b0706-688c-4995-9c15-d8f424f53fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:37.146503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:37.147516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:37.148410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:37.148410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:37.148410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:37.150847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 317/635] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:37.152722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4166cb6-9249-4319-9fba-511c2f14e3d9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.153364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4166cb6-9249-4319-9fba-511c2f14e3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.153839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4166cb6-9249-4319-9fba-511c2f14e3d9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.155638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 318/636] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:37.157309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4166cb6-9249-4319-9fba-511c2f14e3d9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.191295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4ed08-a5b6-451d-8153-0a5b818281d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:37.192815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4ed08-a5b6-451d-8153-0a5b818281d3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:37.295042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4166cb6-9249-4319-9fba-511c2f14e3d9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:37.333420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 319/637] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:37.334600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6f4-13c0-4421-85b3-781039009423 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.334880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6f4-13c0-4421-85b3-781039009423 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.335147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6f4-13c0-4421-85b3-781039009423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.341957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:37.342284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.342504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.342673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.343359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:37.343673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:37.345129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:37.358321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 319/638] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:37.359176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.364062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-0dcbf6f4-13c0-4421-85b3-781039009423 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:37.365314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0dcbf6f4-13c0-4421-85b3-781039009423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:37.431913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:37.450651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:37.468606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 320/639] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:37.468606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf6f4-13c0-4421-85b3-781039009423 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.513678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee4b02-27fc-4bcf-af42-252f3c867dad None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:37.563548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3e25a-3c19-467f-b5c7-a66f8fc8d034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:37.564901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3e25a-3c19-467f-b5c7-a66f8fc8d034 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.565322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3e25a-3c19-467f-b5c7-a66f8fc8d034 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.565727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3e25a-3c19-467f-b5c7-a66f8fc8d034 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.584751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e3e25a-3c19-467f-b5c7-a66f8fc8d034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-77e3e25a-3c19-467f-b5c7-a66f8fc8d034 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:37.593200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:37.594282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 320/640] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:37.595550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3e25a-3c19-467f-b5c7-a66f8fc8d034 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.637142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf6f4-13c0-4421-85b3-781039009423 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:37.689208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:37.690046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.690331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.690595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.704533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:37.707686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:37.730850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:37.733986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:37.737348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:37.737348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:37.737348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:37.745155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3e25a-3c19-467f-b5c7-a66f8fc8d034 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:37.745249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 321/641] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:37.745249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.745385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:37.770636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:37.770636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.771811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.771811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.781142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:37.782966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:37.798144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:37.799227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:37.800461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 321/642] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:37.801568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.811286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:37.894103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fb73a-191d-4dbf-b8fb-816319b094df None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:37.920377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc9df0-c4b3-44ee-aa3e-c79448c8faf4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:37.935770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:37.936182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.936462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.936762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.937425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:37.937727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:37.939109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:37.947806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 322/643] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:37.948426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.961589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:37.962199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.962479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.962748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.963389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:37.963689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:37.965346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:37.982735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 322/644] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:37.982735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.008591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.073637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36a935-27c9-4f14-a9cb-c36daa5e4445 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.077308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.095181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbafc6-1c73-4654-bdf6-7465467cbe30 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.113584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbcb4a-277b-4955-86d1-06ec6a0d3d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.114382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcb4a-277b-4955-86d1-06ec6a0d3d19 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.114527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcb4a-277b-4955-86d1-06ec6a0d3d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.114784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcb4a-277b-4955-86d1-06ec6a0d3d19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.126172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbcb4a-277b-4955-86d1-06ec6a0d3d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-fddbcb4a-277b-4955-86d1-06ec6a0d3d19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:38.131435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:38.132380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 323/645] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:38.137687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbcb4a-277b-4955-86d1-06ec6a0d3d19 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.150336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.150983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.151356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.151724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.153100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:38.153495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.154922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.176417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 323/646] 10.222.0.95 () {56 vars in 1137 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:38.177300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.247858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbcb4a-277b-4955-86d1-06ec6a0d3d19 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.276176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.282870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.283505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.283784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.284053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.296850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.298329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.298437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5fc0d-9fd9-4e8b-8b49-007d28179cb0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:38.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:38.314718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:38.315589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:38.315589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:38.315589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:38.315589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 324/647] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:38.315589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.336921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.337551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.337829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.338095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.340752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.345532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.346978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:38.359283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:38.360389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 324/648] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:38.361782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.440234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237701a6-fd75-41b0-8654-79a9227f7ee1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.477671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.478329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.478547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.478816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.479480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:38.479786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.481371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9bbbd-31e9-4261-b531-6bd2fea15874 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.481846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.494833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 325/649] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:38.494833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.524924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be08f49-295e-4114-a616-c0a25c076744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.525632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be08f49-295e-4114-a616-c0a25c076744 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.525912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be08f49-295e-4114-a616-c0a25c076744 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.526179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be08f49-295e-4114-a616-c0a25c076744 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.539384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.581110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.603187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be08f49-295e-4114-a616-c0a25c076744 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.604684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be08f49-295e-4114-a616-c0a25c076744 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.605217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916a36d-157d-4aa5-b361-f6835ecd4e09 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.640208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.640837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.641113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.641381 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.642019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:38.642318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.643691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.655671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 326/650] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:38.660711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.705470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 325/651] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:38.706471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be08f49-295e-4114-a616-c0a25c076744 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.749790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.764870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6657460-3361-470c-a997-f9c0c829f314 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.802444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabacb3-db7c-4d54-a04b-0bf5c96ca7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.803062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabacb3-db7c-4d54-a04b-0bf5c96ca7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.803339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabacb3-db7c-4d54-a04b-0bf5c96ca7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.803623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabacb3-db7c-4d54-a04b-0bf5c96ca7b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.804304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabacb3-db7c-4d54-a04b-0bf5c96ca7b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:38.804629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabacb3-db7c-4d54-a04b-0bf5c96ca7b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.806071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabacb3-db7c-4d54-a04b-0bf5c96ca7b3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.811283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be08f49-295e-4114-a616-c0a25c076744 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.812797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.818258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 327/652] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:38.819449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2c8a3-97bb-4b45-8601-87ce060dd64b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.819735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2c8a3-97bb-4b45-8601-87ce060dd64b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.820008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2c8a3-97bb-4b45-8601-87ce060dd64b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.821230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 328/653] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:38.821921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2c8a3-97bb-4b45-8601-87ce060dd64b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.851419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.852041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.852316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.852605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.862588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.864382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:38.878922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:38.879891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:38.880900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:38.880900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:38.880900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:38.880900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 326/654] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:38.880900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:38.930517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2c8a3-97bb-4b45-8601-87ce060dd64b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.963305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.963936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.964216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.965222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.972858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.974325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.976232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.985765 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d4913-c500-43cf-9c66-864f69967407 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:38.986775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:38.987750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 329/655] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:38.988760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.022303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b612a2a-f3a7-4677-95a3-05afc6530f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.022930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b612a2a-f3a7-4677-95a3-05afc6530f7c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.023212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b612a2a-f3a7-4677-95a3-05afc6530f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.023488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b612a2a-f3a7-4677-95a3-05afc6530f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.034997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b612a2a-f3a7-4677-95a3-05afc6530f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:39.036440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b612a2a-f3a7-4677-95a3-05afc6530f7c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:39.047345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1b612a2a-f3a7-4677-95a3-05afc6530f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:39.051948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:39.052970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:39.053888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:39.053888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:39.053888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:39.053888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 327/656] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:39.054481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aab6e-2d8e-4bb5-ac64-f187cfa3400e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.054481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aab6e-2d8e-4bb5-ac64-f187cfa3400e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.054739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aab6e-2d8e-4bb5-ac64-f187cfa3400e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.072911 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9f2aab6e-2d8e-4bb5-ac64-f187cfa3400e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:39.074240 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9f2aab6e-2d8e-4bb5-ac64-f187cfa3400e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:39.093533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5f0d3-8dfe-4742-b0e9-51fc905233e9 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:39.127539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.128143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.128419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.128721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.129452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=aaab594854824542bd85426b0f16cbf6) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:39.129758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:39.131199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:39.138498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 330/657] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:38 2022] GET /identity/v3/role_assignments?user.id=aaab594854824542bd85426b0f16cbf6&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:39.139355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.156187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 328/658] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:39.157358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aab6e-2d8e-4bb5-ac64-f187cfa3400e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.243428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:39.272355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aab6e-2d8e-4bb5-ac64-f187cfa3400e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:39.274101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb31d74-8c30-426c-b24d-d84ddc05bdf9 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:39.304972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:39.317942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.319104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.319406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.319629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.319733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.320347 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.320447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:39.320672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.320773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:39.320943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.321579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:39.321879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:39.322098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:39.323241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:39.342471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 329/659] 10.222.0.95 () {56 vars in 1139 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:39.343160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.344305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 331/660] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:39.345271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.467610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee387e-0e2c-49cb-8ea5-d6d077e4206a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:39.471982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bec35-5fe9-40e3-bdc7-2985751e180a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:39.506506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:39.510756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.512122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.512756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.512756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.512902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.513540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.513824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.514096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.514755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:39.515060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:39.516453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:39.521948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:39.524185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:39.529069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 330/661] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:39.529916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:39.540707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:39.541878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:04:39.543098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:39.544127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 332/662] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:39.545078 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.592953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:39.646967 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee33635-143e-4519-b8ec-982b9f67a4af None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:39.656336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60841e0d-a2f9-4549-b7f4-1d8f33b7ab69 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:39.681302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37925-efb0-4cca-8e46-b1313135e42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.681910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37925-efb0-4cca-8e46-b1313135e42f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.682192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37925-efb0-4cca-8e46-b1313135e42f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.682469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37925-efb0-4cca-8e46-b1313135e42f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.691917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effbfeb-3889-448e-b986-72cf5231954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.692982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effbfeb-3889-448e-b986-72cf5231954b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.693258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effbfeb-3889-448e-b986-72cf5231954b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.693530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effbfeb-3889-448e-b986-72cf5231954b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.705820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effbfeb-3889-448e-b986-72cf5231954b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-6effbfeb-3889-448e-b986-72cf5231954b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:39.711076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:39.712013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 333/663] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:39.713623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effbfeb-3889-448e-b986-72cf5231954b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.756144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:39.770232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37925-efb0-4cca-8e46-b1313135e42f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:39.772710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37925-efb0-4cca-8e46-b1313135e42f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:39.849566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effbfeb-3889-448e-b986-72cf5231954b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:39.851211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:39.887953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:39.888597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:39.888955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:39.889221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:39.895910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:39.897412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:39.906136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 331/664] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:39.906136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37925-efb0-4cca-8e46-b1313135e42f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:39.908989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:39.909876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:39.910790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:39.910790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:39.910790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:39.910790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:39.910790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:39.910790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:39.910790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:39.910790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:39.914856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 334/665] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:39.915672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.004273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.014357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37925-efb0-4cca-8e46-b1313135e42f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.022873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa738e0-0945-4459-b609-d2cd04a83d7d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.052387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.053077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.053359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.053657 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.054394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:40.054695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.056125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.060709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.061311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.061588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.061857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.062515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:40.062811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.064626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.078503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 332/666] 10.222.0.95 () {56 vars in 1137 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:40.078503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.079581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 335/667] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:40.081308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.111505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.186922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dff0d-74fd-432b-8249-c27473e5904e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.197125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc120dcc-ab1d-4ace-951a-88949003d012 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.222984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.223609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.223883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.224154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.232645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.233321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.233607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.233878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.234575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:40.234883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.235558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.236311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.237401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.248630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 336/668] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:40.249473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:40.253483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:40.254421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:40.255322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:40.255322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:40.255322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:40.263060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 333/669] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:40.263708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.268942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.343365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.362399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b5a9d-ae59-41ac-9148-3b8bd1354ef8 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.373586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa58a80-28b3-4620-b00a-76c45bd81b6b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.400712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791764e-d6f0-4826-a2cb-da34e19adeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.401329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791764e-d6f0-4826-a2cb-da34e19adeac None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.401611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791764e-d6f0-4826-a2cb-da34e19adeac None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.402474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791764e-d6f0-4826-a2cb-da34e19adeac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.411537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.412365 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.412805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.413271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.425257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.426716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:40.441787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:40.442790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:40.443725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:40.443725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:40.443725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:40.443725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 334/670] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:40.443725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.491951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791764e-d6f0-4826-a2cb-da34e19adeac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.493458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791764e-d6f0-4826-a2cb-da34e19adeac None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.502202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.548889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01def2-a3e4-4b61-a765-8401bd7a3311 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.583811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.584408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.584797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.585076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.585880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=8b6c90a2b95c498494a73b43e7138905) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:40.586175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.587541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.589354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.594909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 335/671] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/role_assignments?user.id=8b6c90a2b95c498494a73b43e7138905&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:40.595636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.619961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 337/672] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:40.621966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791764e-d6f0-4826-a2cb-da34e19adeac None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.742400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39a1e1-0d9a-410b-b9ec-8ff24419478c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.746656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791764e-d6f0-4826-a2cb-da34e19adeac None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.765540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.788624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.789415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.789705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.789976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.790679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:40.790974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.791810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.792322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.793004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.793393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.793666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.794680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:40.794990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.796897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.807290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 336/673] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:40.808225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.812967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 338/674] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:40.817783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:40.875302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:40.931979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f629e-d3cf-4686-a800-4f53f6e119dc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.932721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17113c5-970c-44e1-8e95-c918507f772a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:40.968857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.969498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.969776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.970047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.972165 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:40.972926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:40.973201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:40.973604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:40.977167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.978625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.985732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:40.987185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:40.991190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:40.992081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:40.993021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 339/675] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:40.993506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:41.004819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:41.006972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:41.009198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:41.009198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:41.009198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:41.009198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 337/676] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:41.009198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.045245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.105427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2240565-8a28-4d87-bd43-62997bbf9bac None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.107784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.132708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdc6dd-5c18-4a9e-999b-d283e6f3791d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.150377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.151076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.151372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.151638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.160553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.162053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:41.174881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:41.176091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:41.177366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:41.178317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.179070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.179485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.179887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.180722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 340/677] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:41.181272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.181623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.181896 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.184304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.197974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 338/678] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.199869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.292008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8c82c-8629-4127-893b-273760060f3c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.303237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.312406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03a392-a664-47ba-b2e4-3ebcfd13fa8b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.330210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.330814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.331088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.331356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.332000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.332304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.333931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.352074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 341/679] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.352074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.356189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689e950-7f97-4df1-b7df-d49a85cf05dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.356836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689e950-7f97-4df1-b7df-d49a85cf05dc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.357123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689e950-7f97-4df1-b7df-d49a85cf05dc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.357414 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689e950-7f97-4df1-b7df-d49a85cf05dc None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.358129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689e950-7f97-4df1-b7df-d49a85cf05dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.358432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689e950-7f97-4df1-b7df-d49a85cf05dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.359865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689e950-7f97-4df1-b7df-d49a85cf05dc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.384083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 339/680] 10.222.0.95 () {56 vars in 1135 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.384083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30bee-2004-439d-b971-ca5181508b3b None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.384083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30bee-2004-439d-b971-ca5181508b3b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.384083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30bee-2004-439d-b971-ca5181508b3b None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.384083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 340/681] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:41.384083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c30bee-2004-439d-b971-ca5181508b3b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.385901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.470434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7136b-c231-4a4a-835f-4c24a47417f0 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.497565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c30bee-2004-439d-b971-ca5181508b3b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.507014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.507621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.507903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.508172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.508927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=5f1d5c29df30465ead9834b63c3e93bb) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.509237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.510689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.524611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 342/682] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/role_assignments?user.id=5f1d5c29df30465ead9834b63c3e93bb&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.524611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.546892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaabdf-dbf8-4e5a-9164-0855dda7153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.547690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaabdf-dbf8-4e5a-9164-0855dda7153b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.548048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaabdf-dbf8-4e5a-9164-0855dda7153b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.548382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaabdf-dbf8-4e5a-9164-0855dda7153b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.557423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaabdf-dbf8-4e5a-9164-0855dda7153b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.559458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaabdf-dbf8-4e5a-9164-0855dda7153b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3ceaabdf-dbf8-4e5a-9164-0855dda7153b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:41.572841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:41.573992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:41.575182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 341/683] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:41.575858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae92b7d-068c-401d-87e0-c69310a74af8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.576143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae92b7d-068c-401d-87e0-c69310a74af8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.577672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae92b7d-068c-401d-87e0-c69310a74af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.597126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-bae92b7d-068c-401d-87e0-c69310a74af8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:41.598591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bae92b7d-068c-401d-87e0-c69310a74af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:41.600885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.638342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.656274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4677ff-6990-4370-bc34-b9d81690f3c4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.685188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 342/684] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:41.686479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae92b7d-068c-401d-87e0-c69310a74af8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.697834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53874e1-9d7a-4752-b8dc-683ab2a27834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.699466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53874e1-9d7a-4752-b8dc-683ab2a27834 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.700467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53874e1-9d7a-4752-b8dc-683ab2a27834 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.700873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53874e1-9d7a-4752-b8dc-683ab2a27834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.783777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53874e1-9d7a-4752-b8dc-683ab2a27834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.785597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53874e1-9d7a-4752-b8dc-683ab2a27834 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.812324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae92b7d-068c-401d-87e0-c69310a74af8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.852454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.853499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.853946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.854374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.855451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:41.855925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.857751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.865304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.872732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 343/685] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:41.877519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.890858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.907154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 343/686] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:41.913251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53874e1-9d7a-4752-b8dc-683ab2a27834 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.003567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28751d95-f3fc-4fc0-9849-edb9f6f54d26 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.048489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53874e1-9d7a-4752-b8dc-683ab2a27834 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.052614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.052614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.053014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.053383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.054273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=115298313c1243d282d09a8e8005002c) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:42.054679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.056278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.063382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 344/687] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/role_assignments?user.id=115298313c1243d282d09a8e8005002c&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:42.064718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.117588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.118632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.119128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.120032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.147372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.148827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:42.166557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:42.167669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:42.168605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:42.168605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:42.168605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:42.168782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 344/688] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:42.175800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.224954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.268964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.288729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303cc25-8416-4988-a2b3-dc594f3b7b3f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.332346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144914a8-9515-4eb2-b7cf-612fb0b5b348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.333076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144914a8-9515-4eb2-b7cf-612fb0b5b348 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.333357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144914a8-9515-4eb2-b7cf-612fb0b5b348 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.333632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144914a8-9515-4eb2-b7cf-612fb0b5b348 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.339385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0e052-3e76-4344-b111-d14d4bd073f4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.349343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144914a8-9515-4eb2-b7cf-612fb0b5b348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-144914a8-9515-4eb2-b7cf-612fb0b5b348 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:42.356500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:42.357521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:42.358622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 345/689] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:42.363769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144914a8-9515-4eb2-b7cf-612fb0b5b348 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.379925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.380714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.381156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.381496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.382212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:42.382581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.384221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.399540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 345/690] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:42.406057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.477813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144914a8-9515-4eb2-b7cf-612fb0b5b348 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.502387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.516102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.516850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.517205 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.517489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.517749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.518166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:42.518470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.519857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.520912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c4e26-421d-4975-9e9b-78db2e73e1c8 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.530357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 346/691] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:42.537578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.564795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.565437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.565717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.566159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.579231 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.583670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:42.604778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:42.606443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:42.607339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 346/692] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:42.612636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.719433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1d885-654e-474d-abf0-ef5e083fafc2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.754418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.755029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.755305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.755571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.756319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:42.756640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.758307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.772093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 347/693] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:42.778136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.802718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dc196-0d9e-4272-80fd-9c568fbbcd00 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.809826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.849664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcc9f8-6643-4d13-adb0-228733ef48bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.850747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc9f8-6643-4d13-adb0-228733ef48bb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.851312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc9f8-6643-4d13-adb0-228733ef48bb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.851778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc9f8-6643-4d13-adb0-228733ef48bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.865578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcc9f8-6643-4d13-adb0-228733ef48bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.867351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcc9f8-6643-4d13-adb0-228733ef48bb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-dfdcc9f8-6643-4d13-adb0-228733ef48bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:42.883643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:42.884643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:42.885545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:42.885545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:42.885545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:42.888217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 347/694] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:42.889257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa1e7c-4a56-45d1-b4b6-af76bceddc23 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:42.889937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a2bf2-0dc6-44e3-800d-6e942403fd96 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.890410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a2bf2-0dc6-44e3-800d-6e942403fd96 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.890964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a2bf2-0dc6-44e3-800d-6e942403fd96 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.896524 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 348/695] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:42.897713 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a2bf2-0dc6-44e3-800d-6e942403fd96 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:42.928394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06b179-3d15-45ae-9a40-50705075d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:42.929223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06b179-3d15-45ae-9a40-50705075d40c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.929628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06b179-3d15-45ae-9a40-50705075d40c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.930055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06b179-3d15-45ae-9a40-50705075d40c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.931454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06b179-3d15-45ae-9a40-50705075d40c None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=bac9441fcf60412dbc33e0589a86c8ce) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:42.931996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06b179-3d15-45ae-9a40-50705075d40c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:42.934864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06b179-3d15-45ae-9a40-50705075d40c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:42.942169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 348/696] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/role_assignments?user.id=bac9441fcf60412dbc33e0589a86c8ce&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:42.948272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:42.956664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b41b8-0df3-4694-8a84-91e9405c88e7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:42.956664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b41b8-0df3-4694-8a84-91e9405c88e7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:42.956664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b41b8-0df3-4694-8a84-91e9405c88e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:42.973147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-145b41b8-0df3-4694-8a84-91e9405c88e7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:42.973497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-145b41b8-0df3-4694-8a84-91e9405c88e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:43.052217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 349/697] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:43.073310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a2bf2-0dc6-44e3-800d-6e942403fd96 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.130233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.130862 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.131139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.131405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.132057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:43.132355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.133876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.152742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 349/698] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:43.158468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b41b8-0df3-4694-8a84-91e9405c88e7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.180854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.273109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b41b8-0df3-4694-8a84-91e9405c88e7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.309359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.310034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.310276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.310553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.317875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.319344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:43.331737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:43.332823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:43.333836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 350/699] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:43.340312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.433558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.445904 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.455378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5ea4d-2a5f-4e1c-b76c-13a5c35e6606 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.501927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.503072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.503495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.503907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.505532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:43.506066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.507663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.532688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 350/700] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:43.537835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.546351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.586714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58e950-0fb6-4c36-adfb-796f49d31daa None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.633530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.634272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.634680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.635028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.649281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.651051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.666124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a52517-1557-4aa4-b2eb-83ff1559aa51 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:43.671364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:43.672325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:43.673260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:43.673260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:43.673260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:43.673260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 351/701] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:43.684112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:43.732040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22db4e-0059-47c3-9000-5755563ed38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.732731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22db4e-0059-47c3-9000-5755563ed38a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.733014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22db4e-0059-47c3-9000-5755563ed38a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.733282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22db4e-0059-47c3-9000-5755563ed38a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.782464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.860603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22db4e-0059-47c3-9000-5755563ed38a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.861997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22db4e-0059-47c3-9000-5755563ed38a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.876744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de98ebc-5254-4b5a-9f17-56eb8ca875c6 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:43.921939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:43.938685 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:43.939314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:43.939595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:43.939868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:43.940524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:43.940850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:43.942237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:43.973605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 352/702] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:43.975831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:44.031894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 351/703] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 19:04:44.033233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13db939-b344-449a-b8be-1dc620d2a03c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.033527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13db939-b344-449a-b8be-1dc620d2a03c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.033802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13db939-b344-449a-b8be-1dc620d2a03c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.035334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 352/704] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:44.040340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13db939-b344-449a-b8be-1dc620d2a03c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:44.117231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:44.134608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198584e4-5c7e-4f82-9f65-51e692702626 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:44.172603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13db939-b344-449a-b8be-1dc620d2a03c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:44.197119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451e01b-a287-49ca-bc31-9959b667c2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:44.198370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451e01b-a287-49ca-bc31-9959b667c2df None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.198959 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451e01b-a287-49ca-bc31-9959b667c2df None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.199496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451e01b-a287-49ca-bc31-9959b667c2df None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.200617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451e01b-a287-49ca-bc31-9959b667c2df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:44.203329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451e01b-a287-49ca-bc31-9959b667c2df None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:44.211608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:44.212241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.212523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.212823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.225909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:44.227375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:44.233759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 353/705] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:04:43 2022] POST /identity/v3/users => generated 254 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:44.235450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d4e10-10e2-403f-8f84-ae17eb5b6700 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.235970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d4e10-10e2-403f-8f84-ae17eb5b6700 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.236490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d4e10-10e2-403f-8f84-ae17eb5b6700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:44.247555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:44.248685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:44.249592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:44.249592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:44.249592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:44.249592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 353/706] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:44.262587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:44.263564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-2d8d4e10-10e2-403f-8f84-ae17eb5b6700 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:44.265040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2d8d4e10-10e2-403f-8f84-ae17eb5b6700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:44.346381 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:44.419750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 354/707] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:44.453820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f3b4f-b3c5-42d8-b81a-43ebe22a09e8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:44.466695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:44.495238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c271de3-6c8c-4553-a75f-4b48593de9d4 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.495611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c271de3-6c8c-4553-a75f-4b48593de9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.495890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c271de3-6c8c-4553-a75f-4b48593de9d4 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.497664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 355/708] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:44.502643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a01c8-5dac-489a-b7b2-c873d286501d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.502931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a01c8-5dac-489a-b7b2-c873d286501d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.503209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a01c8-5dac-489a-b7b2-c873d286501d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.504474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:44.505329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.505823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.506305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.507137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:44.507639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:44.509214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:44.523572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 354/709] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:44.527502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6a1a01c8-5dac-489a-b7b2-c873d286501d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:44.528693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:44.528808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6a1a01c8-5dac-489a-b7b2-c873d286501d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:44.627455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 356/710] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:44.648077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c698c4-42e0-46c7-a61c-e90668650296 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:44.685176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:44.686038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.686540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.686950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.694271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:44.695900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:44.708047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:44.709187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:44.710293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:44.710293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:44.710293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:44.710293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:44.710293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:44.710293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:44.710293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:44.710293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:44.712263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 355/711] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:44.718641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a01c8-5dac-489a-b7b2-c873d286501d None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:44.733381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:44.783456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:44.829721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a01c8-5dac-489a-b7b2-c873d286501d None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:44.831597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:44.868132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:44.869133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.869611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.870075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.871769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:44.872281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:44.874034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:44.888681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 357/712] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:44.893223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:44.930910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6cbcf-cb0e-4569-a8f8-cc65e0298d32 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:44.969231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:44.970410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:44.970410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:44.970410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] PATH_INFO: `/v3/services/network` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:44.970987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:44.972364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:44.984706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:44.985802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:44.986734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 356/713] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:44 2022] GET /identity/v3/services/network => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:44.987540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:45.006737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f6735-5569-4641-89af-974d34ad1aa8 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:45.043272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:45.044245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.044772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.045280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.046195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=aaab594854824542bd85426b0f16cbf6) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:45.046715 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:45.048336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:45.055531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 358/714] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:44 2022] GET /identity/v3/role_assignments?user.id=aaab594854824542bd85426b0f16cbf6&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:45.056637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:45.086717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:45.133237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:45.156854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dca0d-5cee-4cb9-b170-b62923341d71 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:45.240356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:45.240917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91b0fc-6668-41e3-bd1d-cdbf7ced1825 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:45.241043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.241216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.241481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.242070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:45.243439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:45.260186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:45.262665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:45.264028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 357/715] 10.222.0.95 () {56 vars in 1129 bytes} [Wed Aug 31 19:04:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:45.269084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:45.291497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:45.292113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.292960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.293781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.293953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] RBAC: Adding query filter params (name=network) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:45.294103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:45.295484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:45.309804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 359/716] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:04:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:45.312810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:45.434416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:45.444871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67201d68-456e-4f99-a3c8-e7878c094a9a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:45.483874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:45.484528 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.485037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.485315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.485952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] RBAC: Adding query filter params (type=network) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:45.486249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:45.487627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:45.496405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 360/717] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:04:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:45.524425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a00ccc-4d63-4d78-bd64-6ec0c96fcfd5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:45.572084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c2bc8-4b0d-40b4-9f86-6dc5fb419b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:45.572783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c2bc8-4b0d-40b4-9f86-6dc5fb419b0d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.573060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c2bc8-4b0d-40b4-9f86-6dc5fb419b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.573326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c2bc8-4b0d-40b4-9f86-6dc5fb419b0d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.574978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c2bc8-4b0d-40b4-9f86-6dc5fb419b0d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:45.574978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c2bc8-4b0d-40b4-9f86-6dc5fb419b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:45.576309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c2bc8-4b0d-40b4-9f86-6dc5fb419b0d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:45.581606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 358/718] 10.222.0.95 () {56 vars in 1124 bytes} [Wed Aug 31 19:04:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:45.587205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:45.733872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9ca40-d0a6-433c-9a27-c536e13d501a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:45.771486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24d15e-7160-4793-b2ce-f3e87c1915c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:45.772024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d15e-7160-4793-b2ce-f3e87c1915c1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.772301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d15e-7160-4793-b2ce-f3e87c1915c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.772590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d15e-7160-4793-b2ce-f3e87c1915c1 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.773225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24d15e-7160-4793-b2ce-f3e87c1915c1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:45.773521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24d15e-7160-4793-b2ce-f3e87c1915c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:45.774920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24d15e-7160-4793-b2ce-f3e87c1915c1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:45.780072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 361/719] 10.222.0.95 () {56 vars in 1124 bytes} [Wed Aug 31 19:04:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:45.880843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ffc4e-c4c6-407e-8e80-3ad3210655f4 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.881179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ffc4e-c4c6-407e-8e80-3ad3210655f4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.881377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ffc4e-c4c6-407e-8e80-3ad3210655f4 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.883024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 359/720] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:45.889643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eecae-efeb-4172-a2dd-a6eb3ee1051e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.890269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eecae-efeb-4172-a2dd-a6eb3ee1051e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.890269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eecae-efeb-4172-a2dd-a6eb3ee1051e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.913128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-141eecae-efeb-4172-a2dd-a6eb3ee1051e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:45.914337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-141eecae-efeb-4172-a2dd-a6eb3ee1051e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:45.936141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:46.008957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 362/721] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:46.391271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ffc4e-c4c6-407e-8e80-3ad3210655f4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.417114 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:46.534496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ffc4e-c4c6-407e-8e80-3ad3210655f4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.585543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d085-507a-4433-8b7c-1d417ef3897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:46.586174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d085-507a-4433-8b7c-1d417ef3897c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.586451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d085-507a-4433-8b7c-1d417ef3897c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.586722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d085-507a-4433-8b7c-1d417ef3897c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.604519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927d085-507a-4433-8b7c-1d417ef3897c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8927d085-507a-4433-8b7c-1d417ef3897c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:46.610597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:46.611550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:46.612393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 360/722] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:46.617688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141eecae-efeb-4172-a2dd-a6eb3ee1051e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.742954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141eecae-efeb-4172-a2dd-a6eb3ee1051e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.789549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5547c91-5810-4458-9a22-36423c07a786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:46.790165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5547c91-5810-4458-9a22-36423c07a786 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.790442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5547c91-5810-4458-9a22-36423c07a786 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.790706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5547c91-5810-4458-9a22-36423c07a786 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.791430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5547c91-5810-4458-9a22-36423c07a786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:46.791728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5547c91-5810-4458-9a22-36423c07a786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:46.793803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5547c91-5810-4458-9a22-36423c07a786 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:46.808920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 363/723] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:46.811767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d085-507a-4433-8b7c-1d417ef3897c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.875204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84aa6-5c2e-448f-90cd-33bb867259ba None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.875520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84aa6-5c2e-448f-90cd-33bb867259ba None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.875719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84aa6-5c2e-448f-90cd-33bb867259ba None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.877205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 364/724] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:46.883583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5394c-4011-474f-9d49-08a9fe913ad6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.884115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5394c-4011-474f-9d49-08a9fe913ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.885116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5394c-4011-474f-9d49-08a9fe913ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.908298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-efa5394c-4011-474f-9d49-08a9fe913ad6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:46.910870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-efa5394c-4011-474f-9d49-08a9fe913ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:46.914716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:46.967962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d085-507a-4433-8b7c-1d417ef3897c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.984981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.004996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.005546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.005827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.006098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.017548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.019353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:47.034226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:47.036116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:47.038036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:47.038036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:47.038036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:47.038036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 361/725] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:47.042033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.083986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 365/726] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 19:04:47.238251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1329f455-1838-4c26-8e25-81554c87feca None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.281395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.287306 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6788c9b-1437-480d-8568-ef61a9ba8790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.289467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788c9b-1437-480d-8568-ef61a9ba8790 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.289751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788c9b-1437-480d-8568-ef61a9ba8790 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.290021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6788c9b-1437-480d-8568-ef61a9ba8790 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.290666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6788c9b-1437-480d-8568-ef61a9ba8790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:47.290963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6788c9b-1437-480d-8568-ef61a9ba8790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.292487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6788c9b-1437-480d-8568-ef61a9ba8790 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.310937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 362/727] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:47.316355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5394c-4011-474f-9d49-08a9fe913ad6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.349728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3e557-977e-474c-be0e-beb4d62be60d None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.350107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3e557-977e-474c-be0e-beb4d62be60d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.350274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3e557-977e-474c-be0e-beb4d62be60d None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.351872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 363/728] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:47.357380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2df0b-93ac-4fb9-b183-711363b6b826 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.357545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2df0b-93ac-4fb9-b183-711363b6b826 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.357738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2df0b-93ac-4fb9-b183-711363b6b826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.389671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a9b2df0b-93ac-4fb9-b183-711363b6b826 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:47.390940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a9b2df0b-93ac-4fb9-b183-711363b6b826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:47.445341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5394c-4011-474f-9d49-08a9fe913ad6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.481031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 364/729] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:47.482330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2df0b-93ac-4fb9-b183-711363b6b826 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.486986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.487694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.488041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.488383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.497375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.498937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.514841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:47.523275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:47.524275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:47.525299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 366/730] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:47.532359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.663949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2df0b-93ac-4fb9-b183-711363b6b826 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.701547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.710326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a689a7-11bd-44be-aed5-563191757c43 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.751562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.752236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.752529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.752829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.753521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:47.753823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.755261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.758133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e5c86-3d64-4aff-918c-bee5c15935e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.758769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5c86-3d64-4aff-918c-bee5c15935e5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.759048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5c86-3d64-4aff-918c-bee5c15935e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.759313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5c86-3d64-4aff-918c-bee5c15935e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.779329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e5c86-3d64-4aff-918c-bee5c15935e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-246e5c86-3d64-4aff-918c-bee5c15935e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:47.784232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:47.786394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:47.788393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:47.794652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 365/731] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:47.794652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e5c86-3d64-4aff-918c-bee5c15935e5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.800531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 367/732] 10.222.0.95 () {56 vars in 1135 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:47.805576 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.880215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:47.921344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e5c86-3d64-4aff-918c-bee5c15935e5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.937105 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7e5f3-7d82-4d6d-8ece-1ace7bd38a3b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:47.962734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.963342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.963620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.963891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:47.965242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:47.965583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:47.966989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:47.978411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 366/733] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:47.979257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:47.989327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a9442-2bc1-4635-a3b9-21401194fb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:47.990072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a9442-2bc1-4635-a3b9-21401194fb1e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:47.990457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a9442-2bc1-4635-a3b9-21401194fb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:47.990845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a9442-2bc1-4635-a3b9-21401194fb1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.039887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.079909 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a9442-2bc1-4635-a3b9-21401194fb1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.081897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a9442-2bc1-4635-a3b9-21401194fb1e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.099617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d46c44-24fe-4507-9216-6d8b16cddb10 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.146642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a2a9c-99f4-4ed7-a97b-39f5fc578d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.147574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a2a9c-99f4-4ed7-a97b-39f5fc578d19 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.147574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a2a9c-99f4-4ed7-a97b-39f5fc578d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.147860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a2a9c-99f4-4ed7-a97b-39f5fc578d19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.152200 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.162240 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a2a9c-99f4-4ed7-a97b-39f5fc578d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-314a2a9c-99f4-4ed7-a97b-39f5fc578d19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:48.168174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:48.169340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 367/734] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:48.170279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a2a9c-99f4-4ed7-a97b-39f5fc578d19 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.224209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 368/735] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 19:04:48.226037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9e5db-45f0-4bcd-8b3c-67e21c97c69e None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.226395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9e5db-45f0-4bcd-8b3c-67e21c97c69e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.226742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9e5db-45f0-4bcd-8b3c-67e21c97c69e None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.228189 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 369/736] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:48.228839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9e5db-45f0-4bcd-8b3c-67e21c97c69e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.298216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a2a9c-99f4-4ed7-a97b-39f5fc578d19 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.299914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.333984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.334611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.334891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.335161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.336513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9e5db-45f0-4bcd-8b3c-67e21c97c69e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.346287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.347760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:48.361972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:48.363011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:48.363948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:48.363948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:48.363948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:48.363948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 368/737] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:48.363948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.373540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babeb0d-1e98-4059-97d2-b5ca5cdc5dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.374154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babeb0d-1e98-4059-97d2-b5ca5cdc5dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.374436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babeb0d-1e98-4059-97d2-b5ca5cdc5dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.374709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babeb0d-1e98-4059-97d2-b5ca5cdc5dd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.375403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babeb0d-1e98-4059-97d2-b5ca5cdc5dd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:48.375705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babeb0d-1e98-4059-97d2-b5ca5cdc5dd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.377169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babeb0d-1e98-4059-97d2-b5ca5cdc5dd5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.385416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 370/738] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.387309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89f1b-8321-4235-8dd3-c67fff5effbd None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.387597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89f1b-8321-4235-8dd3-c67fff5effbd None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.387870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89f1b-8321-4235-8dd3-c67fff5effbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.407011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f1a89f1b-8321-4235-8dd3-c67fff5effbd None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:48.408215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f1a89f1b-8321-4235-8dd3-c67fff5effbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:48.427819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.472957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5a46d-51b6-4ae8-a0f8-4c46c36518ce None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.483258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 371/739] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:48.484523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66a0aa-4972-428f-8dd9-04b7e4286275 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.484841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66a0aa-4972-428f-8dd9-04b7e4286275 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.485115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66a0aa-4972-428f-8dd9-04b7e4286275 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.486491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 372/740] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:48.487037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66a0aa-4972-428f-8dd9-04b7e4286275 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.509218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.510444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.510866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.511797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.525283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.527715 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:48.543273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:48.544436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:48.545557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:48.545557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:48.545557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:48.546946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 369/741] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:48.547963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.558839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.622194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66a0aa-4972-428f-8dd9-04b7e4286275 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.658569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c1875-58a1-4b5c-a161-d804fef5a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.659177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c1875-58a1-4b5c-a161-d804fef5a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.659457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c1875-58a1-4b5c-a161-d804fef5a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.659728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c1875-58a1-4b5c-a161-d804fef5a1f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.660365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c1875-58a1-4b5c-a161-d804fef5a1f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:48.660692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c1875-58a1-4b5c-a161-d804fef5a1f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.662072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c1875-58a1-4b5c-a161-d804fef5a1f1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.668492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bac65-f5a9-41bd-8def-517c033e889e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.673738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 373/742] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.675462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b090-cd9e-4714-9ae6-8100d560fb94 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.675942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b090-cd9e-4714-9ae6-8100d560fb94 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.676406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242b090-cd9e-4714-9ae6-8100d560fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.697433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f242b090-cd9e-4714-9ae6-8100d560fb94 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:48.698918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f242b090-cd9e-4714-9ae6-8100d560fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:48.702444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.706659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.707545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.707545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.707822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.720098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.721583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:48.736688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:48.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:48.738608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:48.738608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:48.738608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:48.742497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 370/743] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:48.743369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.773790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 374/744] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:04:48.774652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242b090-cd9e-4714-9ae6-8100d560fb94 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.808402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.852227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b9bbd-beb6-4a66-a661-c6d6c0522f53 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.891156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.891370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242b090-cd9e-4714-9ae6-8100d560fb94 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:48.891764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.892043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.892311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.892980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:48.893281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.895661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.912831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 371/745] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:48.912831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:48.941176 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:48.941865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:48.942147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:48.942415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:48.951214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:48.952839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:48.955825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:48.967427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:48.968512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 375/746] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:48.974087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.033440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03845c6-7db8-4dee-a668-d480e9f00422 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.056083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.075491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.076099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.076378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.076685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.077330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:49.077631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.079024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.084400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c5e5b-fd79-4bd9-abf2-49a45b007cd1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.091418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 372/747] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:49.092252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.123109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9271b88-d89a-490e-a196-0b79ffc42d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.123754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9271b88-d89a-490e-a196-0b79ffc42d3a None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.123998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9271b88-d89a-490e-a196-0b79ffc42d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.124817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9271b88-d89a-490e-a196-0b79ffc42d3a None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.125066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9271b88-d89a-490e-a196-0b79ffc42d3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.126450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9271b88-d89a-490e-a196-0b79ffc42d3a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.145519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 376/748] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:04:48 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:49.146401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9271b88-d89a-490e-a196-0b79ffc42d3a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.211005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c86e-0bce-4bfe-821e-d8b04082a3ca None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.239072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.269135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.270526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.271059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.271543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.280290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.282071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.287007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9271b88-d89a-490e-a196-0b79ffc42d3a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:49.294906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:49.296010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 373/749] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:49.297009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.332729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.333514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.333883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.334250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.335083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:49.335452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.336939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.352022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.358286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 377/750] 10.222.0.95 () {56 vars in 1149 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:49.359181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.435162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4fd9-fef1-4cad-a76f-62b6a70be597 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.472548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef722b-7529-4994-b2da-1b072864ad91 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.474448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50e827-741f-4c41-9cb4-3798bea101ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.475257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50e827-741f-4c41-9cb4-3798bea101ff None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.475535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50e827-741f-4c41-9cb4-3798bea101ff None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.475808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50e827-741f-4c41-9cb4-3798bea101ff None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.476458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50e827-741f-4c41-9cb4-3798bea101ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.477860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50e827-741f-4c41-9cb4-3798bea101ff None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.493676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.508725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 374/751] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:04:49 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:49.508725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50e827-741f-4c41-9cb4-3798bea101ff None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.519168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.519813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.520093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.520420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.532821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.534327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:49.548325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:49.549666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:49.550848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 378/752] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:49.551629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.661935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.678463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50e827-741f-4c41-9cb4-3798bea101ff None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.690623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9aa04-6913-4afe-bc14-f40ee7624a28 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.733904 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.734799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.735265 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.735790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.737453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:49.737575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.741225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.741591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddc3b0-a8dc-42af-8222-09dde1c1dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.742533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddc3b0-a8dc-42af-8222-09dde1c1dc75 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.742975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddc3b0-a8dc-42af-8222-09dde1c1dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.743412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddc3b0-a8dc-42af-8222-09dde1c1dc75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.760764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 375/753] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:49.763140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.813296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.855401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddc3b0-a8dc-42af-8222-09dde1c1dc75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.858016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddc3b0-a8dc-42af-8222-09dde1c1dc75 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.961937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ffb23-35fa-45d9-b071-0a93b1b73cd4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.002667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.003335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.003611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.003884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.005116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:50.005430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.007156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.021629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 379/754] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:04:50.023186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddc3b0-a8dc-42af-8222-09dde1c1dc75 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.036100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 376/755] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:50.037218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.056336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:50.143659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:50.181851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddc3b0-a8dc-42af-8222-09dde1c1dc75 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.187040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11c21-d4cf-47eb-84c4-26104acd7647 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.222626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.223400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.223773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.224142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.225022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=bac9441fcf60412dbc33e0589a86c8ce) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:50.225427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.227248 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.233234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.234108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 380/756] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/role_assignments?user.id=bac9441fcf60412dbc33e0589a86c8ce&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:50.235059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.236704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.243843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.244227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.271626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.273196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:50.295939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:50.297007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:50.297916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:50.297916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:50.297916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:50.303148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 377/757] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:50.305738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.385435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:50.429772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bec0a-0cd8-4518-b468-e7b2e53be6b4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.472226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.473316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.473847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.474312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.475147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:50.475649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.477239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.492074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 378/758] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:50.498470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.522799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d243a8-02ac-4fba-8845-81c1d2f96acc None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.582394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.582854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.583133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.583402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.591776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:50.597394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.598884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.609412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820ced-137c-49e1-a951-e4183c36b632 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:50.628054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:50.629096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:50.630027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:50.630027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:50.630027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:50.630027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 381/759] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:50.634793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.638091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:50.650300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.651260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.652355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.652852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.667976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.671166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:50.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:50.695538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:50.696561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 379/760] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:50.705583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.830413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5a5c1-159c-4d96-8051-838ab381de46 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.848500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d0a08-eff6-4be5-8573-c1b9009d7c6a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:50.876186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.877211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.877702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.878188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.879078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:50.879577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.881742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.909943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 380/761] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:50.918485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:50.919428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:50.919964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:50.921470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:50.935061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:50.936176 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:50.936710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:50.938318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:50.961365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 382/762] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:50.967437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:51.015308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:51.073484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:51.100018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586ae6-bbd7-4891-addf-f243624590ea None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:51.140529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:51.141642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.142198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.142664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.143719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=8a1dcd805548450cb3411c62083a4bc5) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:51.144221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:51.145884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:51.153255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 381/763] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/role_assignments?user.id=8a1dcd805548450cb3411c62083a4bc5&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:51.179458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8386603-d3de-4d57-b1db-3399d5916748 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:51.242037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804c98b-2cbb-428e-8ac2-c4111cbbaf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:51.242696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804c98b-2cbb-428e-8ac2-c4111cbbaf2c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.242977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804c98b-2cbb-428e-8ac2-c4111cbbaf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.243251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804c98b-2cbb-428e-8ac2-c4111cbbaf2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.257840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804c98b-2cbb-428e-8ac2-c4111cbbaf2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:51.259356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804c98b-2cbb-428e-8ac2-c4111cbbaf2c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-b804c98b-2cbb-428e-8ac2-c4111cbbaf2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:51.279497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:51.280510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:51.281445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 383/764] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:51.286575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:51.408751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093034d-39a3-498a-989a-d605f045ef5a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:51.419175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:51.428328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aac2b-893f-4450-8196-29c551b5bc4d None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.428665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aac2b-893f-4450-8196-29c551b5bc4d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.428896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aac2b-893f-4450-8196-29c551b5bc4d None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.432829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 384/765] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:51.437423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34945d57-2824-445a-992c-48e68f338f98 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.437704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34945d57-2824-445a-992c-48e68f338f98 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.437976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34945d57-2824-445a-992c-48e68f338f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.457194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-34945d57-2824-445a-992c-48e68f338f98 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:51.458380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-34945d57-2824-445a-992c-48e68f338f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:51.462269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:51.462895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.463175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.463447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.464907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:51.465307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:51.466944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:51.491418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 382/766] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:51.497460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:51.577583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 385/767] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:04:51.661018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811d243-4dae-4566-b2c3-1b343f5b91a5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:51.713370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859a42f-bfd4-4976-90f4-4c128ba5d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:51.715531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a42f-bfd4-4976-90f4-4c128ba5d22c None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.716412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a42f-bfd4-4976-90f4-4c128ba5d22c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.717089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a42f-bfd4-4976-90f4-4c128ba5d22c None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/aaab594854824542bd85426b0f16cbf6/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.719417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859a42f-bfd4-4976-90f4-4c128ba5d22c None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=aaab594854824542bd85426b0f16cbf6, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:51.721107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859a42f-bfd4-4976-90f4-4c128ba5d22c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:51.740509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 383/768] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:51 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/aaab594854824542bd85426b0f16cbf6/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:04:51.894044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6eca-17c4-443f-b05c-5d1d4bc5c498 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.894695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6eca-17c4-443f-b05c-5d1d4bc5c498 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.895195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6eca-17c4-443f-b05c-5d1d4bc5c498 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.897164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 386/769] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:51.908668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45826328-ced0-4d25-9fee-4db186694acf None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:51.908972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45826328-ced0-4d25-9fee-4db186694acf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:51.909216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45826328-ced0-4d25-9fee-4db186694acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:51.938571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-45826328-ced0-4d25-9fee-4db186694acf None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:51.939725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-45826328-ced0-4d25-9fee-4db186694acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:51.969865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6eca-17c4-443f-b05c-5d1d4bc5c498 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:51.974548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.031989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.043845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 384/770] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:04:52.097488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6eca-17c4-443f-b05c-5d1d4bc5c498 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.135235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.136417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.136800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.137138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] PATH_INFO: `/v3/services/network` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.137858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.139248 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:52.150838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:52.152802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:52.154614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 387/771] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:51 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:52.163838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45826328-ced0-4d25-9fee-4db186694acf None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.302336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45826328-ced0-4d25-9fee-4db186694acf None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.343565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.344550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.345068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.345533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.346347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] RBAC: Adding query filter params (name=network) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:52.346855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.348413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.355190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 385/772] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:52.360593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.391913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.430761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.522300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c52f-a2dd-42f5-9341-7ce0599c5d76 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.562750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ee7a6-4c89-4dde-b143-778f5faf4109 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.563344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456fdb-073b-4e3e-8ebe-bf2dd8ee5fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.564345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456fdb-073b-4e3e-8ebe-bf2dd8ee5fbd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.565121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.565640 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456fdb-073b-4e3e-8ebe-bf2dd8ee5fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.566131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456fdb-073b-4e3e-8ebe-bf2dd8ee5fbd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.567528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92456fdb-073b-4e3e-8ebe-bf2dd8ee5fbd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.569174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92456fdb-073b-4e3e-8ebe-bf2dd8ee5fbd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-92456fdb-073b-4e3e-8ebe-bf2dd8ee5fbd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:52.580755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:52.581913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:52.582780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:52.585612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 386/773] 10.222.0.95 () {56 vars in 1129 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:52.592768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc721ca-f779-4472-8a91-304c480f09b9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.593364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc721ca-f779-4472-8a91-304c480f09b9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.594164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc721ca-f779-4472-8a91-304c480f09b9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.596773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 387/774] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:52.597829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc721ca-f779-4472-8a91-304c480f09b9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.612354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ac8c2-1c86-4944-8409-3c1161cf6007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.613070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ac8c2-1c86-4944-8409-3c1161cf6007 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.613770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ac8c2-1c86-4944-8409-3c1161cf6007 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.614059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ac8c2-1c86-4944-8409-3c1161cf6007 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.614701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ac8c2-1c86-4944-8409-3c1161cf6007 None admin] RBAC: Adding query filter params (type=network) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:52.615144 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ac8c2-1c86-4944-8409-3c1161cf6007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.616746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ac8c2-1c86-4944-8409-3c1161cf6007 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.624523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 388/775] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:52.626352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13a75-ec9f-4437-a68b-da72a58e25fc None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.626655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13a75-ec9f-4437-a68b-da72a58e25fc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.626932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc13a75-ec9f-4437-a68b-da72a58e25fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.652994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4dc13a75-ec9f-4437-a68b-da72a58e25fc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:52.654529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4dc13a75-ec9f-4437-a68b-da72a58e25fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:52.707823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.727395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc721ca-f779-4472-8a91-304c480f09b9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.756595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 389/776] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:04:52.756595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc13a75-ec9f-4437-a68b-da72a58e25fc None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.778063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.779019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.779504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.780066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.780934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:52.781451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.783076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.806594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 388/777] 10.222.0.95 () {56 vars in 1124 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:52.813522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:52.870378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:52.874914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc13a75-ec9f-4437-a68b-da72a58e25fc None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:52.939669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:52.940283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:52.940557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:52.940867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:52.943799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] RBAC: Adding query filter params (interface=public, service_id=dfe53ee6392f4df0a10c5de1126f6700, region_id=RegionOne) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:52.944112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:52.945545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:52.962678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 390/778] 10.222.0.95 () {56 vars in 1251 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/endpoints?service_id=dfe53ee6392f4df0a10c5de1126f6700&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:52.971567 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.046709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940b10d-be99-4a7d-af30-39542a624692 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.107717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.109064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.109831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.110727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.112543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:53.113599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.116959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.127739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996f63b-8ca7-4dc0-baaa-de9bf37a4f6b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.134683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 389/779] 10.222.0.95 () {56 vars in 1124 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:53.137163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.161550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.180991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8fa5e-7bca-4750-b4ad-ee005d7faf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.181886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8fa5e-7bca-4750-b4ad-ee005d7faf1b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.182240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8fa5e-7bca-4750-b4ad-ee005d7faf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.183179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8fa5e-7bca-4750-b4ad-ee005d7faf1b None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.184492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8fa5e-7bca-4750-b4ad-ee005d7faf1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.187111 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8fa5e-7bca-4750-b4ad-ee005d7faf1b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.203632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 391/780] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:04:52 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:53.204966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8fa5e-7bca-4750-b4ad-ee005d7faf1b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.282557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.337372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a26d9-d498-45e2-b758-b53f64828d8d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.411319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b380982-ba97-46da-9084-257138a6233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.411878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380982-ba97-46da-9084-257138a6233b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.412160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380982-ba97-46da-9084-257138a6233b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.412432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380982-ba97-46da-9084-257138a6233b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.428032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8fa5e-7bca-4750-b4ad-ee005d7faf1b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.428473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b380982-ba97-46da-9084-257138a6233b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-0b380982-ba97-46da-9084-257138a6233b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:53.435890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:53.436938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:53.438192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 390/781] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:53.444614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b380982-ba97-46da-9084-257138a6233b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.476687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.477682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.478177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.478672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.480244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] RBAC: Adding query filter params (interface=public, service_id=d5a9170f9e604351ab2643be134acdea, region_id=RegionOne) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:53.481377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.483846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.490995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 392/782] 10.222.0.95 () {56 vars in 1251 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/endpoints?service_id=d5a9170f9e604351ab2643be134acdea&interface=public®ion_id=RegionOne => generated 194 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:53.497236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.542541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.587259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b380982-ba97-46da-9084-257138a6233b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.642777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.649461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c155e27-50b1-457f-95cd-2eed2c7e7ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.650238 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155e27-50b1-457f-95cd-2eed2c7e7ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.650527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155e27-50b1-457f-95cd-2eed2c7e7ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.650800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155e27-50b1-457f-95cd-2eed2c7e7ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.652136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c155e27-50b1-457f-95cd-2eed2c7e7ee0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:53.652862 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c155e27-50b1-457f-95cd-2eed2c7e7ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.654391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c155e27-50b1-457f-95cd-2eed2c7e7ee0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.654808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9fc76-9be1-4997-a272-a1607b1060b6 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.665602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 391/783] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:53.666869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6ab5c-96c5-4ebc-8ce0-c97eade85508 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.667743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6ab5c-96c5-4ebc-8ce0-c97eade85508 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.668164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6ab5c-96c5-4ebc-8ce0-c97eade85508 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.669565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 392/784] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:53.673585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6ab5c-96c5-4ebc-8ce0-c97eade85508 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.701956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e133874-24c4-4f95-b385-31cbe2cd8b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.702670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133874-24c4-4f95-b385-31cbe2cd8b0c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.702957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133874-24c4-4f95-b385-31cbe2cd8b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.703228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133874-24c4-4f95-b385-31cbe2cd8b0c None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.703843 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e133874-24c4-4f95-b385-31cbe2cd8b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.705277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e133874-24c4-4f95-b385-31cbe2cd8b0c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.711466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 393/785] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:53.712835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb673-590d-4a5b-a7aa-e2bced0d17c2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.713220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb673-590d-4a5b-a7aa-e2bced0d17c2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.713525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affb673-590d-4a5b-a7aa-e2bced0d17c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.746682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-9affb673-590d-4a5b-a7aa-e2bced0d17c2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:53.747928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9affb673-590d-4a5b-a7aa-e2bced0d17c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:53.813903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6ab5c-96c5-4ebc-8ce0-c97eade85508 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:53.857709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:53.858742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.859079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.859353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.863406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.874546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:53.876321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:53.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:53.896991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:53.899042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:53.899042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:53.899042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:53.899042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 393/786] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:53.904382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:53.933463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 394/787] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:04:54.038919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa7ab5-1a20-41ba-bdf1-82486dedd44a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.060592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578f2e3-6ba8-4441-a77b-3643720b082a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.060905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578f2e3-6ba8-4441-a77b-3643720b082a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.061160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578f2e3-6ba8-4441-a77b-3643720b082a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.062658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 395/788] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:04:54.068485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dadfb7-69fe-4303-88c4-b5815de307b8 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.068801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dadfb7-69fe-4303-88c4-b5815de307b8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.069077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dadfb7-69fe-4303-88c4-b5815de307b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.088452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.089813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.090181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.090654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.091119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c6dadfb7-69fe-4303-88c4-b5815de307b8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:54.091454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:54.091762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.092814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c6dadfb7-69fe-4303-88c4-b5815de307b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:54.093355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.111190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 394/789] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:54.116920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.192852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 396/790] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:04:54.249999 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.290269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dadfb7-69fe-4303-88c4-b5815de307b8 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.309931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.333777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d1fe-5724-4ac2-8832-83c8a0f65917 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.393757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.394693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.394970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.395241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.410300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.411776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:54.438802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:54.439822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:54.440801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:54.448746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 395/791] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:54.448746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.465051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dadfb7-69fe-4303-88c4-b5815de307b8 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.528670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a464cd-8907-4ef3-a80e-1025b8f2839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.529707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a464cd-8907-4ef3-a80e-1025b8f2839e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.530487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a464cd-8907-4ef3-a80e-1025b8f2839e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.530978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a464cd-8907-4ef3-a80e-1025b8f2839e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.546293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a464cd-8907-4ef3-a80e-1025b8f2839e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-31a464cd-8907-4ef3-a80e-1025b8f2839e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:54.553919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:54.554888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:04:54.555719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:54.559928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 397/792] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:04:54.561678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a464cd-8907-4ef3-a80e-1025b8f2839e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.571045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41738437-0c9c-456d-b306-dbb6310bb2e8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.582423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.618328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.619082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.619802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.620486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.621578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:54.621991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.623689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.644924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 396/793] 10.222.0.95 () {56 vars in 1135 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:54.646080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.659655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.701411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a464cd-8907-4ef3-a80e-1025b8f2839e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.747125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e3af-31cf-4da0-aa68-eb7cc5adae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.747762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e3af-31cf-4da0-aa68-eb7cc5adae7d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.748040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e3af-31cf-4da0-aa68-eb7cc5adae7d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.748308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e3af-31cf-4da0-aa68-eb7cc5adae7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.762323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34e3af-31cf-4da0-aa68-eb7cc5adae7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:54.763468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2a44f-a215-4bf2-84d1-1972dcd427ee None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-6d34e3af-31cf-4da0-aa68-eb7cc5adae7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:54.768199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:54.769164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:54.770016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:54.774215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 398/794] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:54.775509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e3af-31cf-4da0-aa68-eb7cc5adae7d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.805596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.806573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.807073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.807587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.809232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:54.809741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.811332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.820991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 397/795] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:54.822551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.853327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.904798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e3af-31cf-4da0-aa68-eb7cc5adae7d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.939174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cbb4c-63ad-4a08-bd2f-78a6a4bb92f4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.952870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.953428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.953708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.953975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.956295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.969150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.971490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.989957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.991226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.991705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.992199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.993151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:54.993652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.995847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:54.999778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:55.000792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:55.001742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:55.001742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:55.001742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:55.002343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 399/796] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:55.003541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.011963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 398/797] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.013220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.138204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.155647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fa45e-da45-4982-8975-82b2d50062bc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.160229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6f781-04b9-4ccc-a28a-e7331390931a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.199946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.200776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.201005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.201271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.207459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.208349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.208975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.209227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.209891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:55.210196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.211590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.215695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.217628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.225785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 399/798] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.226754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:55.237481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:55.238491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:55.239475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:55.239475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:55.239475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:55.242087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 400/799] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:55.243402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.299565 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.369868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bae26e-c091-420f-8924-4ee61982c18e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.371062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495947c-0492-4061-8ba1-c002dc7c411a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.408490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.409264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.409598 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.410051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.410167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.410564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.410676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.410996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.419500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.421071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.423664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.425622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:55.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:55.436170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:55.437304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:55.437304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:55.437304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:55.437304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:55.437304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:55.437304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:55.437304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:55.437304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:55.440675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 401/800] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:55.440675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:55.450143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:55.451134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:55.452080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:55.452080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:55.452080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:55.456384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 400/801] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:55.457425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.460381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.555159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.561665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305bffa-b103-4288-8deb-bb4a20c57645 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.576030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680c14-ee36-47fa-ac52-998e85a546c0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.600620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.601237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.601519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.601795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.602448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:55.602757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.604171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.613528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db4f66-5690-4f0b-a33a-4b661cc47ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.614138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db4f66-5690-4f0b-a33a-4b661cc47ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.614534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db4f66-5690-4f0b-a33a-4b661cc47ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.614807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db4f66-5690-4f0b-a33a-4b661cc47ec9 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.615497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db4f66-5690-4f0b-a33a-4b661cc47ec9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:55.615802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db4f66-5690-4f0b-a33a-4b661cc47ec9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.617261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db4f66-5690-4f0b-a33a-4b661cc47ec9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.623552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 402/802] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.623552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.641701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 401/803] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.643055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb6c3a-1dd0-4aea-a594-d9739afe9e6b None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.643342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb6c3a-1dd0-4aea-a594-d9739afe9e6b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.643613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb6c3a-1dd0-4aea-a594-d9739afe9e6b None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.645026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 402/804] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:55.645589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb6c3a-1dd0-4aea-a594-d9739afe9e6b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.716696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.737434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b2799-dcc3-4d93-912a-f9568e9391e2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.763838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb6c3a-1dd0-4aea-a594-d9739afe9e6b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.776906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.777564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.777835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.778105 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.778744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:55.779045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.780420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.795548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 403/805] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:55.796348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.807485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a4d44-51ed-4b81-8dae-ebdeac0f7a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.809260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4d44-51ed-4b81-8dae-ebdeac0f7a37 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.809543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4d44-51ed-4b81-8dae-ebdeac0f7a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.809808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4d44-51ed-4b81-8dae-ebdeac0f7a37 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.817376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a4d44-51ed-4b81-8dae-ebdeac0f7a37 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.818890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a4d44-51ed-4b81-8dae-ebdeac0f7a37 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:55.820313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-bb2a4d44-51ed-4b81-8dae-ebdeac0f7a37 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:55.831401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:55.836443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 403/806] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9537b2-b9ba-440b-a378-0bf184469203 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.839452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9537b2-b9ba-440b-a378-0bf184469203 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.840420 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9537b2-b9ba-440b-a378-0bf184469203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.857750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2b9537b2-b9ba-440b-a378-0bf184469203 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:55.858923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2b9537b2-b9ba-440b-a378-0bf184469203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:55.931286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9bd2-16b2-42f5-ba21-26787e3b788c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:55.940661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 404/807] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:04:55.940661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9537b2-b9ba-440b-a378-0bf184469203 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:55.971466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed86618-aba5-45ad-842c-737c6e1d4f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:55.972167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed86618-aba5-45ad-842c-737c6e1d4f19 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:55.972449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed86618-aba5-45ad-842c-737c6e1d4f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:55.972744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed86618-aba5-45ad-842c-737c6e1d4f19 None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/bac9441fcf60412dbc33e0589a86c8ce/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:55.975038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed86618-aba5-45ad-842c-737c6e1d4f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=bac9441fcf60412dbc33e0589a86c8ce, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:55.976802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed86618-aba5-45ad-842c-737c6e1d4f19 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.001218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 404/808] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:55 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/bac9441fcf60412dbc33e0589a86c8ce/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:56.001218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed86618-aba5-45ad-842c-737c6e1d4f19 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.031946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.096490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9537b2-b9ba-440b-a378-0bf184469203 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.123327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.143952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892591f-244b-4b49-b240-d64c07728f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.145099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892591f-244b-4b49-b240-d64c07728f5d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.145682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892591f-244b-4b49-b240-d64c07728f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.146159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892591f-244b-4b49-b240-d64c07728f5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.156469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed86618-aba5-45ad-842c-737c6e1d4f19 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.157031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892591f-244b-4b49-b240-d64c07728f5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.158970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892591f-244b-4b49-b240-d64c07728f5d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f892591f-244b-4b49-b240-d64c07728f5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:56.174981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:56.176098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:56.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 405/809] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:56.179307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e70cbb-74a4-455b-80f0-a1ad01692a67 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.179585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e70cbb-74a4-455b-80f0-a1ad01692a67 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.179854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e70cbb-74a4-455b-80f0-a1ad01692a67 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.181831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 406/810] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:56.185453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e70cbb-74a4-455b-80f0-a1ad01692a67 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.203483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d469f9-96a1-40af-b97c-2912a97ff1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.204150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d469f9-96a1-40af-b97c-2912a97ff1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.204368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d469f9-96a1-40af-b97c-2912a97ff1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.204660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d469f9-96a1-40af-b97c-2912a97ff1c3 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.205399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d469f9-96a1-40af-b97c-2912a97ff1c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:56.205697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d469f9-96a1-40af-b97c-2912a97ff1c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.207161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d469f9-96a1-40af-b97c-2912a97ff1c3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.238938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 405/811] 10.222.0.95 () {56 vars in 1149 bytes} [Wed Aug 31 19:04:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:56.240766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d92d-5df8-4f5a-93ec-7a1cedcca2ea None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.241062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d92d-5df8-4f5a-93ec-7a1cedcca2ea None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.241379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d92d-5df8-4f5a-93ec-7a1cedcca2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.262327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d5e9d92d-5df8-4f5a-93ec-7a1cedcca2ea None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:56.263564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d5e9d92d-5df8-4f5a-93ec-7a1cedcca2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:56.303996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e70cbb-74a4-455b-80f0-a1ad01692a67 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.318704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.350033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.350460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.350720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.351041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.356771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 406/812] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:04:56.356952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:56.356952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.357117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9d92d-5df8-4f5a-93ec-7a1cedcca2ea None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.357215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.384891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 407/813] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:04:56.386009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.431579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.476702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9d92d-5df8-4f5a-93ec-7a1cedcca2ea None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.514792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc3cf-d6f8-4604-83f4-6e15194fd6da None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.517617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.518571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.518987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.519330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] PATH_INFO: `/v3/services/network` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.519988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.521480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:56.531687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:56.532789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:56.533646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 407/814] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:56.538669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.558460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.559952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.560505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.560975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.572057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.575990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:56.595492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:56.596519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:56.597458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:56.597458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:56.597458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:56.604106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 408/815] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:56.606098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.640771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.667506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52457af7-b7a3-48d9-b296-050c7feb0a81 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.696922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:56.705151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf9dd6-885f-491d-8e04-bf49ca500231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.705769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf9dd6-885f-491d-8e04-bf49ca500231 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.706045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf9dd6-885f-491d-8e04-bf49ca500231 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.706317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf9dd6-885f-491d-8e04-bf49ca500231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.738916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e074-1deb-4ce0-9f0f-000d517e377b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:56.782617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:56.784192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:56.784733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:56.785406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:56.785870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf9dd6-885f-491d-8e04-bf49ca500231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.786352 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] RBAC: Adding query filter params (name=network) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:56.786822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:56.787360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf9dd6-885f-491d-8e04-bf49ca500231 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.788542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:56.801073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 409/816] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:56.801697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.918646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 408/817] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:04:56.919969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf9dd6-885f-491d-8e04-bf49ca500231 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:56.953693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.010025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.027652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846c6c-c177-4208-b65c-785cd2b7121d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.033133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf9dd6-885f-491d-8e04-bf49ca500231 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.063785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.064410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.064718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.064975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.065634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:57.065929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.067314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.075052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.076105 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.076448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.076905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.077758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.079659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.086572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 410/818] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:57.087665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:57.089743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:57.090706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:04:57.091645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:57.095667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 409/819] 10.222.0.95 () {56 vars in 1129 bytes} [Wed Aug 31 19:04:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:57.096561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.209459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca92d8b-5c4c-41c0-9206-d174a30c39f0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.219749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.228155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f0cd9-af4c-40cb-a70d-1d2c18f56c58 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.249106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.249716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.249996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.250268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.251369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] RBAC: Adding query filter params (type=network) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:57.251740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.253146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.263477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 411/820] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:57.264216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.267888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.268637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.268915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.269179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.279983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.282618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.284467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:57.301064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:57.302084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:57.303029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:57.303029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:57.303029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:04:57.303029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 410/821] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:57.303029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.398161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6a08-6873-4b8d-a715-f8cc250eea90 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.423161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58babd-7575-4870-a59e-9116b7bcb6df None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.438589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.439332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.439677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.440018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.447125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.449160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:57.461092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:57.462060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:57.462951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 412/822] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:57.463437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.468701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.469334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.469615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.469888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.470538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:57.470842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.472245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.477334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 411/823] 10.222.0.95 () {56 vars in 1124 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:57.478221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.493253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.553779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.593584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e37187-2460-4f7e-bdce-5b40ff3a8a0f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.604831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10ff0-cc38-42bd-b628-9efc5e9b2c05 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.638244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.638875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.639167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.639440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.640090 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:57.640394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.641838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.647313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d69dd-db0b-4cb0-bf33-42bf69a60833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.648100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d69dd-db0b-4cb0-bf33-42bf69a60833 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.648530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d69dd-db0b-4cb0-bf33-42bf69a60833 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.649031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d69dd-db0b-4cb0-bf33-42bf69a60833 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.650175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d69dd-db0b-4cb0-bf33-42bf69a60833 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.652077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d69dd-db0b-4cb0-bf33-42bf69a60833 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.659797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 412/824] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:57.659797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.684609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 413/825] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:04:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:57.684609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d69dd-db0b-4cb0-bf33-42bf69a60833 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.759791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.806308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f72f9-9cad-4f6b-ad11-2772c5cbb683 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.835852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d69dd-db0b-4cb0-bf33-42bf69a60833 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:57.865542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.866526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.867033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.867511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.868428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:57.868972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.871915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:57.877582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.883394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:57.884142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:57.884674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:57.885020 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:57.887929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:57.887929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:57.888425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:57.893755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 414/826] 10.222.0.95 () {56 vars in 1124 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:57.895395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:57.906602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 413/827] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:57.907548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:58.030026 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e2562-8716-46ab-a6b0-07ad0e95b945 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.043590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e33558-6e8a-4304-af65-585a42e5809d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.063164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:58.070079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810084d6-7fed-48d7-aa91-8797880f3ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.070684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810084d6-7fed-48d7-aa91-8797880f3ab1 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.070966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810084d6-7fed-48d7-aa91-8797880f3ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.071230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810084d6-7fed-48d7-aa91-8797880f3ab1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.071904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810084d6-7fed-48d7-aa91-8797880f3ab1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:58.073854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810084d6-7fed-48d7-aa91-8797880f3ab1 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:58.086142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a5d04-2a63-4e75-8e45-5eb50c17c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.087304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a5d04-2a63-4e75-8e45-5eb50c17c677 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.087988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a5d04-2a63-4e75-8e45-5eb50c17c677 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.088471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a5d04-2a63-4e75-8e45-5eb50c17c677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.098421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a5d04-2a63-4e75-8e45-5eb50c17c677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:58.099882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a5d04-2a63-4e75-8e45-5eb50c17c677 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:58.101342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 414/828] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:04:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:04:58.102143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810084d6-7fed-48d7-aa91-8797880f3ab1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d81a5d04-2a63-4e75-8e45-5eb50c17c677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:04:58.113689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:58.114680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:58.115688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:04:58.115688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:04:58.115688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:04:58.115688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:04:58.115688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:04:58.115688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:04:58.115688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:04:58.115688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:58.120612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 415/829] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:58.122232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd811f64-7bc4-445b-b513-f17ea456a65c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.122514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd811f64-7bc4-445b-b513-f17ea456a65c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.122906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd811f64-7bc4-445b-b513-f17ea456a65c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.124885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 416/830] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:58.125502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd811f64-7bc4-445b-b513-f17ea456a65c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:58.194456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:58.251634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810084d6-7fed-48d7-aa91-8797880f3ab1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.269108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd811f64-7bc4-445b-b513-f17ea456a65c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.301427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b51fa-6194-42b1-9253-3f62f94442ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.302093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b51fa-6194-42b1-9253-3f62f94442ab None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.302370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b51fa-6194-42b1-9253-3f62f94442ab None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.302640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b51fa-6194-42b1-9253-3f62f94442ab None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/8a1dcd805548450cb3411c62083a4bc5/roles/c49d2fa980524a9f95d78b7fff997d1c` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.315909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab350f-71d1-46af-8edc-1cbadbc4222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.317074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab350f-71d1-46af-8edc-1cbadbc4222d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.317410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab350f-71d1-46af-8edc-1cbadbc4222d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.317987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab350f-71d1-46af-8edc-1cbadbc4222d None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.318696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab350f-71d1-46af-8edc-1cbadbc4222d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:58.319054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab350f-71d1-46af-8edc-1cbadbc4222d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:58.319648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b51fa-6194-42b1-9253-3f62f94442ab None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=8a1dcd805548450cb3411c62083a4bc5, role_id=c49d2fa980524a9f95d78b7fff997d1c)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:58.320799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab350f-71d1-46af-8edc-1cbadbc4222d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:58.321194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b51fa-6194-42b1-9253-3f62f94442ab None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:58.334385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 417/831] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:58.335627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e25c46-e032-47da-8e54-9d5da3b8d592 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.335908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e25c46-e032-47da-8e54-9d5da3b8d592 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.336176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e25c46-e032-47da-8e54-9d5da3b8d592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.336896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 415/832] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:04:58 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/8a1dcd805548450cb3411c62083a4bc5/roles/c49d2fa980524a9f95d78b7fff997d1c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:04:58.345671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b51fa-6194-42b1-9253-3f62f94442ab None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:58.362723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-98e25c46-e032-47da-8e54-9d5da3b8d592 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:58.365310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-98e25c46-e032-47da-8e54-9d5da3b8d592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:58.389854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:58.478573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b51fa-6194-42b1-9253-3f62f94442ab None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.479285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 418/833] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:04:58.520187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.520871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.521152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.521486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.522217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=aaab594854824542bd85426b0f16cbf6) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:58.522517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:58.524129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:58.531741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 416/834] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:04:58 2022] GET /identity/v3/role_assignments?user.id=aaab594854824542bd85426b0f16cbf6&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:58.763367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e25c46-e032-47da-8e54-9d5da3b8d592 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:58.800996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:58.874093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e25c46-e032-47da-8e54-9d5da3b8d592 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:58.912173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1ab85-8a61-4e6e-9f56-d6acc76255a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:58.912848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1ab85-8a61-4e6e-9f56-d6acc76255a7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:58.913131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1ab85-8a61-4e6e-9f56-d6acc76255a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:58.913403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1ab85-8a61-4e6e-9f56-d6acc76255a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:58.925409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1ab85-8a61-4e6e-9f56-d6acc76255a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a5d1ab85-8a61-4e6e-9f56-d6acc76255a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:04:58.930333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:58.931290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:58.932603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 419/835] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:58.938455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.053631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f5aa8-9384-439d-9eb4-92daa37125a5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.068957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.095970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.096610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.096896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.097168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.097905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:59.098209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.099598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.110151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 417/836] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:59.115691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1ab85-8a61-4e6e-9f56-d6acc76255a7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.232863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.240384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1ab85-8a61-4e6e-9f56-d6acc76255a7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.279552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.280537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.280993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.281414 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.292830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.294534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:04:59.312403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:59.313625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:04:59.314749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:04:59.314749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:04:59.314749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:59.320735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 420/837] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:59.323011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.436994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bd643-2e95-4c6b-a586-567784c7f417 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.481169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.482253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.482804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.483291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.484324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:59.484817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.486797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.502070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 418/838] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:59.507561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.622892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab180b4c-c884-4862-80d7-8c5e7ee0faea None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.659534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.660561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.661117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.661596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.670857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.672686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:04:59.685898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:04:59.686894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:04:59.687832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:04:59.690952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 421/839] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:04:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:04:59.696963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.717799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.809786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409038a-e966-4d3d-9ca9-a6712d3c10bb None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:59.846122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3569e7-0494-4a33-af2f-9818d4d65ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:59.846729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3569e7-0494-4a33-af2f-9818d4d65ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.847006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3569e7-0494-4a33-af2f-9818d4d65ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.847271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3569e7-0494-4a33-af2f-9818d4d65ce9 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.848001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3569e7-0494-4a33-af2f-9818d4d65ce9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:04:59.848298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3569e7-0494-4a33-af2f-9818d4d65ce9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:59.849779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3569e7-0494-4a33-af2f-9818d4d65ce9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:59.869254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 419/840] 10.222.0.95 () {56 vars in 1135 bytes} [Wed Aug 31 19:04:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:04:59.875847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:59.889741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:59.988834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3693be5a-8fd4-4cfb-893d-8b8b3ddef786 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.025345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d57af-4ffc-40e5-a50a-0c56a9ef684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.027031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d57af-4ffc-40e5-a50a-0c56a9ef684f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.027619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d57af-4ffc-40e5-a50a-0c56a9ef684f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.028128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d57af-4ffc-40e5-a50a-0c56a9ef684f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.105079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d57af-4ffc-40e5-a50a-0c56a9ef684f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.106902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d57af-4ffc-40e5-a50a-0c56a9ef684f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.117781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.202742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ecaec-b0a5-4f74-a638-3f7f078d146a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.203119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ecaec-b0a5-4f74-a638-3f7f078d146a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.203119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ecaec-b0a5-4f74-a638-3f7f078d146a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.204745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 420/841] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:00.212791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e5e8-f31e-4d5e-bebb-22a426c54b20 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.213581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e5e8-f31e-4d5e-bebb-22a426c54b20 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.215514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e5e8-f31e-4d5e-bebb-22a426c54b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.240712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 422/842] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:04:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:00.245421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d57af-4ffc-40e5-a50a-0c56a9ef684f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.249957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-1785e5e8-f31e-4d5e-bebb-22a426c54b20 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:00.251510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1785e5e8-f31e-4d5e-bebb-22a426c54b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:00.352598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 421/843] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:00.369229 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d57af-4ffc-40e5-a50a-0c56a9ef684f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.382119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.408963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e21cdc-667d-4535-a698-3be645558f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.409612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e21cdc-667d-4535-a698-3be645558f5e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.409898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e21cdc-667d-4535-a698-3be645558f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.410175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e21cdc-667d-4535-a698-3be645558f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.421685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e21cdc-667d-4535-a698-3be645558f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.423217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e21cdc-667d-4535-a698-3be645558f5e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-19e21cdc-667d-4535-a698-3be645558f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:00.440008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:00.441056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:00.441977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:00.441977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:00.441977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:00.441977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 423/844] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:00.450360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785e5e8-f31e-4d5e-bebb-22a426c54b20 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.459722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.496628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b4828-f139-486a-b291-af23faa20392 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.496628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b4828-f139-486a-b291-af23faa20392 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.496628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b4828-f139-486a-b291-af23faa20392 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.497097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 424/845] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:00.504477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ddf66-9d1d-4ed5-985c-062291bfb629 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.505071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ddf66-9d1d-4ed5-985c-062291bfb629 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.505536 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ddf66-9d1d-4ed5-985c-062291bfb629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.524446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9f7ddf66-9d1d-4ed5-985c-062291bfb629 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:00.526258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9f7ddf66-9d1d-4ed5-985c-062291bfb629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:00.576110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785e5e8-f31e-4d5e-bebb-22a426c54b20 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.621277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 425/846] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:00.624622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.624798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.625048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.625249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.625906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:00.626208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.627614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.641285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 422/847] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:00.646884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ddf66-9d1d-4ed5-985c-062291bfb629 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.662947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.746599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.758028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:00.769337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ddf66-9d1d-4ed5-985c-062291bfb629 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.787827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278b53e-d3bc-49d4-924a-5184020f2c66 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.812465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcf8df-a7d6-4f88-9baa-a4013096bf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.813124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcf8df-a7d6-4f88-9baa-a4013096bf6f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.813399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcf8df-a7d6-4f88-9baa-a4013096bf6f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.813804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcf8df-a7d6-4f88-9baa-a4013096bf6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.820946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcf8df-a7d6-4f88-9baa-a4013096bf6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.822345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcf8df-a7d6-4f88-9baa-a4013096bf6f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.832022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501639d5-5af7-432b-bfd0-1f28bedc7f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:00.832684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501639d5-5af7-432b-bfd0-1f28bedc7f99 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.832967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501639d5-5af7-432b-bfd0-1f28bedc7f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.833238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501639d5-5af7-432b-bfd0-1f28bedc7f99 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.833820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501639d5-5af7-432b-bfd0-1f28bedc7f99 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:00.835199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501639d5-5af7-432b-bfd0-1f28bedc7f99 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-14bcf8df-a7d6-4f88-9baa-a4013096bf6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:00.838083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:00.839651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 426/848] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:00.841108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafddc34-61db-4cd4-9e99-604fc9db3f0c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.841759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafddc34-61db-4cd4-9e99-604fc9db3f0c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.841759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafddc34-61db-4cd4-9e99-604fc9db3f0c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.842990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 427/849] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:00.847568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafddc34-61db-4cd4-9e99-604fc9db3f0c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-501639d5-5af7-432b-bfd0-1f28bedc7f99 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:00.852079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:00.854735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:00.857822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:00.859964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 423/850] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:00.860393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5feb9-3a4b-44f0-8aa4-bb6375638156 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:00.860842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5feb9-3a4b-44f0-8aa4-bb6375638156 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:00.861318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5feb9-3a4b-44f0-8aa4-bb6375638156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:00.880505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-04f5feb9-3a4b-44f0-8aa4-bb6375638156 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:00.881830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-04f5feb9-3a4b-44f0-8aa4-bb6375638156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:00.954057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafddc34-61db-4cd4-9e99-604fc9db3f0c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:00.955163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 424/851] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:00.956002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5feb9-3a4b-44f0-8aa4-bb6375638156 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.005529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.006105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.006410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.006705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.007447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.007756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.009269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.022313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.022689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 428/852] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:01.023528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.038922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.070258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5feb9-3a4b-44f0-8aa4-bb6375638156 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.111703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.112309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.112606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.112882 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.113515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.113815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.115194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.119871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 425/853] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:01.122866 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.145412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cea3-898d-4c21-8860-d6325a155c39 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.189095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22871df8-1f33-41d3-9dd3-d3cd65e4d080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.190143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871df8-1f33-41d3-9dd3-d3cd65e4d080 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.190686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871df8-1f33-41d3-9dd3-d3cd65e4d080 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.190958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871df8-1f33-41d3-9dd3-d3cd65e4d080 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.204129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22871df8-1f33-41d3-9dd3-d3cd65e4d080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-22871df8-1f33-41d3-9dd3-d3cd65e4d080 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:01.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:01.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:01.210971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 429/854] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:01.211687 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22871df8-1f33-41d3-9dd3-d3cd65e4d080 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.242151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967cb5c-8300-450d-823e-b7efabe76b7d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.270841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.282789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.283398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.283675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.283945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.284696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=bac9441fcf60412dbc33e0589a86c8ce) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.285002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.286439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.287293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.294097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 426/855] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/role_assignments?user.id=bac9441fcf60412dbc33e0589a86c8ce&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:01.294798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.342915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22871df8-1f33-41d3-9dd3-d3cd65e4d080 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.384586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.385222 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.385497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.385962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.387200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.387530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.389428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.395145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 430/856] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:01.395839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.439467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae81688-bf17-4e1c-9996-44c5c75abc29 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.491269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.492753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.492753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.492753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.493165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.494562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.495516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.506663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 427/857] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:01.512908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.532760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a3d21-969e-4339-9c0c-0feb1662528a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.580661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.586926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.587641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.587919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.588187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.588828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.591161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.593722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:01.604487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:01.605625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:01.606643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:01.607600 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 431/858] 10.222.0.95 () {56 vars in 1132 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:01.612676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.675585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4c7b9-5f19-4188-a8b2-66cf82deee92 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.745164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.746607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.747171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.747785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.769765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.771922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.787194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9495de-58bf-47dc-967f-ce05915790be None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:01.796084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:01.797341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:01.798256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:01.798256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:01.798256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:01.800675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 428/859] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:01.809820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.835946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.836590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.836969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.837255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.837897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.838199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:01.840913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:01.848763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 432/860] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:01.856241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:01.944349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acd12e-3d63-46f9-ba7b-8aabfff7253a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:01.946138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.947703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:01.996432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:01.997529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:01.998024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:01.998503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:01.999372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:01.999839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112198b-9bb6-4c0f-bf0d-5d280c6a817f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:02.000142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.001944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.021722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 429/861] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:02.023001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:02.037118 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:02.037889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:02.038401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:02.038583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:02.039235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:02.039539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.041025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.052709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 433/862] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:02.161564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af55ea5-4235-498e-bdbf-1e799054c542 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:02.202328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:02.204311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:02.204311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:02.204311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:02.211457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.212972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:02.225704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:02.226677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:02.227609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 430/863] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:05:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:02.233607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:02.350480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23070ca-ff6f-4c90-aa07-5870a26d8ccd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:02.379012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:02.389793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:02.390769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:02.391470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:02.392034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:02.393188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:02.393561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.395519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.416951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 434/864] 10.222.0.95 () {56 vars in 1149 bytes} [Wed Aug 31 19:05:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:02.423068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:02.452776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:02.545936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388248aa-3dff-47a2-bd28-d6993cf33764 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:02.586024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32ee25-3ecf-44cb-94b5-8cd97df12fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:02.587145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32ee25-3ecf-44cb-94b5-8cd97df12fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:02.587607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32ee25-3ecf-44cb-94b5-8cd97df12fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:02.588035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32ee25-3ecf-44cb-94b5-8cd97df12fb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:02.663783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32ee25-3ecf-44cb-94b5-8cd97df12fb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.665520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32ee25-3ecf-44cb-94b5-8cd97df12fb2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.692978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:02.786538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 431/865] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:02 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:02.792769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:02.910495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53533c9b-f837-4d78-8114-870ba28b7e8b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:02.955284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:02.955833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:02.956115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:02.956383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:02.967594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:02.969151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:02.985590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:02.986622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:02.987582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:02.987582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:02.987582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:02.987582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 435/866] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:02.992994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32ee25-3ecf-44cb-94b5-8cd97df12fb2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.107427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32ee25-3ecf-44cb-94b5-8cd97df12fb2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:03.134765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:03.147192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14408fc3-225f-4982-9c3d-4cfe9a0446e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:03.147831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408fc3-225f-4982-9c3d-4cfe9a0446e5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.148356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408fc3-225f-4982-9c3d-4cfe9a0446e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.149068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408fc3-225f-4982-9c3d-4cfe9a0446e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.150155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14408fc3-225f-4982-9c3d-4cfe9a0446e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:03.151347 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14408fc3-225f-4982-9c3d-4cfe9a0446e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:03.152917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14408fc3-225f-4982-9c3d-4cfe9a0446e5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:03.165790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 432/867] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:03.172758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.214700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:03.306449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6649ede-3e60-4877-aa20-f334bb473973 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:03.350099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:03.350733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.351013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.351279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.359323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:03.360984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:03.365272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458e56-8f6e-42de-ab9d-40120b88399f None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.365560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458e56-8f6e-42de-ab9d-40120b88399f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.365834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458e56-8f6e-42de-ab9d-40120b88399f None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.367392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 433/868] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:03.373536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2d9d-925d-4983-a2cc-fb60948f3c5c None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.373821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2d9d-925d-4983-a2cc-fb60948f3c5c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.374101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2d9d-925d-4983-a2cc-fb60948f3c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:03.377446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:03.378357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:03.379252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 436/869] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:03.388540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.400598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-7b0a2d9d-925d-4983-a2cc-fb60948f3c5c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:03.402464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7b0a2d9d-925d-4983-a2cc-fb60948f3c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:03.494198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 434/870] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:03.497305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4e154-2da1-4d56-9fd3-78cfb9309483 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.497761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4e154-2da1-4d56-9fd3-78cfb9309483 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.498103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4e154-2da1-4d56-9fd3-78cfb9309483 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.500230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 435/871] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:03.507186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd37cd-6cda-4778-ad8c-c88f031174a3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.507640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd37cd-6cda-4778-ad8c-c88f031174a3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.508065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd37cd-6cda-4778-ad8c-c88f031174a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.510324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:03.518422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44686ec-b447-40b4-8892-2e29d5b51a19 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:03.532100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-15cd37cd-6cda-4778-ad8c-c88f031174a3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:03.533656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-15cd37cd-6cda-4778-ad8c-c88f031174a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:03.574049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:03.574821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.575385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.575635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.576519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:03.576962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:03.578510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:03.599756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 437/872] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:03.601078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.652014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 436/873] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:03.738183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe690b-3459-4d36-8846-4b2c9d8edc7f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:03.784724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:03.784724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.784724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.784724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.785197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=8a1dcd805548450cb3411c62083a4bc5) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:03.785197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:03.785197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:03.791581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd37cd-6cda-4778-ad8c-c88f031174a3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.796174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 438/874] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:03 2022] GET /identity/v3/role_assignments?user.id=8a1dcd805548450cb3411c62083a4bc5&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:03.961689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:03.990328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.002782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.075259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd37cd-6cda-4778-ad8c-c88f031174a3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.085143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef8df6-0334-4292-bb1f-58e6905d60ae None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.118547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35faea5-5b0d-44a9-b481-04ed0a3b17a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.124989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35faea5-5b0d-44a9-b481-04ed0a3b17a0 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.125399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35faea5-5b0d-44a9-b481-04ed0a3b17a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.125672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35faea5-5b0d-44a9-b481-04ed0a3b17a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.126346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35faea5-5b0d-44a9-b481-04ed0a3b17a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.127787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35faea5-5b0d-44a9-b481-04ed0a3b17a0 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.129587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4cb44-08e1-4508-b19e-7723bfd82719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.130203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4cb44-08e1-4508-b19e-7723bfd82719 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.130483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4cb44-08e1-4508-b19e-7723bfd82719 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.130754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4cb44-08e1-4508-b19e-7723bfd82719 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.131419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4cb44-08e1-4508-b19e-7723bfd82719 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.132799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4cb44-08e1-4508-b19e-7723bfd82719 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.150918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 439/875] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:05:03 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-c35faea5-5b0d-44a9-b481-04ed0a3b17a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:04.153128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:04.154510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:04.155441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 437/876] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:04.162662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4cb44-08e1-4508-b19e-7723bfd82719 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.255589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786423a-4cce-469f-a8f2-838abbbc4e24 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.255589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786423a-4cce-469f-a8f2-838abbbc4e24 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.255589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786423a-4cce-469f-a8f2-838abbbc4e24 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.256540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 438/877] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:04.262508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5440ae2-14cb-48c1-926a-2c4f47f5d34d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.262836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5440ae2-14cb-48c1-926a-2c4f47f5d34d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.263034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5440ae2-14cb-48c1-926a-2c4f47f5d34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.275581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.285939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e5440ae2-14cb-48c1-926a-2c4f47f5d34d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:04.287469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e5440ae2-14cb-48c1-926a-2c4f47f5d34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:04.314350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4cb44-08e1-4508-b19e-7723bfd82719 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.365302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.366383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.367674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.367674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.367674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:04.368042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.369849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.376124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 440/878] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:04.381969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.400591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 439/879] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:04.502120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c82ea-8fa1-4e92-b88f-c5c233a9ecf9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.537631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb761ef-7df4-4174-bfbc-1158e974a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.538241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb761ef-7df4-4174-bfbc-1158e974a89d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.538526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb761ef-7df4-4174-bfbc-1158e974a89d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.538798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb761ef-7df4-4174-bfbc-1158e974a89d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.539495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb761ef-7df4-4174-bfbc-1158e974a89d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:04.539796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb761ef-7df4-4174-bfbc-1158e974a89d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.541475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb761ef-7df4-4174-bfbc-1158e974a89d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.547367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 441/880] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:04.706884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5440ae2-14cb-48c1-926a-2c4f47f5d34d None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:04.748939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:04.822690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5440ae2-14cb-48c1-926a-2c4f47f5d34d None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:04.860641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a6feb-64ce-4c34-9007-76447157c396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:04.861698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a6feb-64ce-4c34-9007-76447157c396 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:04.862173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a6feb-64ce-4c34-9007-76447157c396 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:04.862646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a6feb-64ce-4c34-9007-76447157c396 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:04.863504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a6feb-64ce-4c34-9007-76447157c396 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:04.865154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a6feb-64ce-4c34-9007-76447157c396 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:04.892797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 440/881] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:05:04 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:06.197669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dcb51-9e50-4598-bff8-eb415949f32e None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.198774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dcb51-9e50-4598-bff8-eb415949f32e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.199052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dcb51-9e50-4598-bff8-eb415949f32e None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.200820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 442/882] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:06.205972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70618ac-4d52-42a1-afdc-42f115673272 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.206678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70618ac-4d52-42a1-afdc-42f115673272 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.206678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70618ac-4d52-42a1-afdc-42f115673272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.227230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e70618ac-4d52-42a1-afdc-42f115673272 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:06.228399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e70618ac-4d52-42a1-afdc-42f115673272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:06.335294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 441/883] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:06.373091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb755-36d9-463f-a859-e2d9a76ba554 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.373427 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb755-36d9-463f-a859-e2d9a76ba554 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.373708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb755-36d9-463f-a859-e2d9a76ba554 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.375353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 443/884] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:06.381831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9215-8354-40fd-af3d-eae981e65453 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.382184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9215-8354-40fd-af3d-eae981e65453 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.382623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f9215-8354-40fd-af3d-eae981e65453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.404858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e15f9215-8354-40fd-af3d-eae981e65453 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:06.406081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e15f9215-8354-40fd-af3d-eae981e65453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:06.426634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:06.492331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 442/885] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:06.659381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cb755-36d9-463f-a859-e2d9a76ba554 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.663768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:06.691477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745cf67-d336-4ba6-b519-baa246571eea None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.691949 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745cf67-d336-4ba6-b519-baa246571eea None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.692398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745cf67-d336-4ba6-b519-baa246571eea None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.694311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 443/886] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:06.700084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57443d2-2234-499f-857d-42977f1d6de7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.700405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57443d2-2234-499f-857d-42977f1d6de7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.700820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57443d2-2234-499f-857d-42977f1d6de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.723091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d57443d2-2234-499f-857d-42977f1d6de7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:06.724999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d57443d2-2234-499f-857d-42977f1d6de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:06.804393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cb755-36d9-463f-a859-e2d9a76ba554 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:06.832524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 444/887] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:06.833839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57443d2-2234-499f-857d-42977f1d6de7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.853605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f77e9-7cdb-4b1d-b8b5-e3be08a0f575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:06.855819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f77e9-7cdb-4b1d-b8b5-e3be08a0f575 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:06.856275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f77e9-7cdb-4b1d-b8b5-e3be08a0f575 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:06.856587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f77e9-7cdb-4b1d-b8b5-e3be08a0f575 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:06.874708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f77e9-7cdb-4b1d-b8b5-e3be08a0f575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-897f77e9-7cdb-4b1d-b8b5-e3be08a0f575 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:06.881651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:06.882656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:06.883514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:06.889049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 444/888] 10.222.0.95 () {56 vars in 1120 bytes} [Wed Aug 31 19:05:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:06.891389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f77e9-7cdb-4b1d-b8b5-e3be08a0f575 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:06.955348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:06.982766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57443d2-2234-499f-857d-42977f1d6de7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:06.987990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.032143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af33211-fc29-4f35-bb65-2d399c75f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.032407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f77e9-7cdb-4b1d-b8b5-e3be08a0f575 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.032878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33211-fc29-4f35-bb65-2d399c75f233 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.033150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33211-fc29-4f35-bb65-2d399c75f233 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.033423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af33211-fc29-4f35-bb65-2d399c75f233 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.034021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af33211-fc29-4f35-bb65-2d399c75f233 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.035411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af33211-fc29-4f35-bb65-2d399c75f233 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-7af33211-fc29-4f35-bb65-2d399c75f233 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:07.045244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:07.046304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:07.047143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 445/889] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:07.048454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f19bbf-eb3e-495c-a939-cc6804a36af2 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.048454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f19bbf-eb3e-495c-a939-cc6804a36af2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.048666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f19bbf-eb3e-495c-a939-cc6804a36af2 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.050308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 446/890] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:07.053790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f19bbf-eb3e-495c-a939-cc6804a36af2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.077900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec17820-9e6d-4f34-a2e4-68a1ebefe960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.078542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec17820-9e6d-4f34-a2e4-68a1ebefe960 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.078891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec17820-9e6d-4f34-a2e4-68a1ebefe960 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.079166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec17820-9e6d-4f34-a2e4-68a1ebefe960 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.079872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec17820-9e6d-4f34-a2e4-68a1ebefe960 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:07.080266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec17820-9e6d-4f34-a2e4-68a1ebefe960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.081805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec17820-9e6d-4f34-a2e4-68a1ebefe960 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.091254 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 445/891] 10.222.0.95 () {56 vars in 1115 bytes} [Wed Aug 31 19:05:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:07.092518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621c885-30f3-4bb7-aabf-c5490780a185 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.092835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621c885-30f3-4bb7-aabf-c5490780a185 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.093109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621c885-30f3-4bb7-aabf-c5490780a185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.112987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0621c885-30f3-4bb7-aabf-c5490780a185 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:07.114955 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0621c885-30f3-4bb7-aabf-c5490780a185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:07.195103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f19bbf-eb3e-495c-a939-cc6804a36af2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.211179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 446/892] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:07.211179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621c885-30f3-4bb7-aabf-c5490780a185 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.239046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.239746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.239970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.240202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.240876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:07.241179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.242779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.254201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 447/893] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:07.254201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.267861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.312207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.361067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621c885-30f3-4bb7-aabf-c5490780a185 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.390136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d70f8c-c8ba-4555-8857-49ed2ecd50bf None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.403040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.403971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.404246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.404611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.417586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.419629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.432556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3e7e8-70d5-4067-9c87-edb80ba7499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.433514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3e7e8-70d5-4067-9c87-edb80ba7499f None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.433992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3e7e8-70d5-4067-9c87-edb80ba7499f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.434454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3e7e8-70d5-4067-9c87-edb80ba7499f None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.435311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3e7e8-70d5-4067-9c87-edb80ba7499f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:07.435468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:07.436447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:07.442793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:07.442793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:07.442793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:07.442793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 447/894] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:07.442793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.443109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3e7e8-70d5-4067-9c87-edb80ba7499f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.456919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 448/895] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:05:07 2022] POST /identity/v3/services => generated 241 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:07.458196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3e7e8-70d5-4067-9c87-edb80ba7499f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.560951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.587695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb85e01-aa4d-4718-8cec-b7b202d8f80b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.600913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.609454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3e7e8-70d5-4067-9c87-edb80ba7499f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.631456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.632355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.632830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.633125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.633869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:07.634215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.636674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.643698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 448/896] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:07.644602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.656384 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.657675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.657675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.657979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.658415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:07.658719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.660119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.674839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 449/897] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:07.680069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.768672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9552ed-267a-4772-89de-6bc88332f4e9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.800141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61166d6a-20e0-47ae-8327-ffdcce250dc3 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.805093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.805722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.806002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.806272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.806856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.809099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:07.825198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:07.826193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:07.827545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 449/898] 10.222.0.95 () {56 vars in 1132 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:07.828360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.834121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.844856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:07.845558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:07.845840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:07.846107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:07.846822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] RBAC: Adding query filter params (interface=public, service_id=14e1130072834789baed6bc41efd35c3, region_id=RegionOne) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:07.847123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:07.848518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:07.861208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 450/899] 10.222.0.95 () {56 vars in 1251 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/endpoints?service_id=14e1130072834789baed6bc41efd35c3&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:07.861208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:07.880719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:07.966467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091aae09-5a10-4043-8e60-3d295e5c959e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:07.991006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9e11b-795a-4bc4-9ec2-df632eeb38d0 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.006949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.007595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.007878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.008181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.017903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.019695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:08.034286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:08.036453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:05:08.038707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:08.040005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.040005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.040005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.040005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.040736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:08.040736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.041001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 450/900] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:08.041001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.043752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.049503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 451/901] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:08.050426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.123900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.175103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.178398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ecc39-2ba9-4a2c-8c8b-0ef56baf683b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.181423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1c82d-ae5e-497c-bc5e-4a51054fe6bc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.215213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.216090 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.216366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.216665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.217397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:08.217699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.219131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.219242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b57faa-700d-48da-a776-7d773803feb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.219943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b57faa-700d-48da-a776-7d773803feb1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.220217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b57faa-700d-48da-a776-7d773803feb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.220489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b57faa-700d-48da-a776-7d773803feb1 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.221214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b57faa-700d-48da-a776-7d773803feb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.222693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b57faa-700d-48da-a776-7d773803feb1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.240749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 451/902] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:05:08.240749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b57faa-700d-48da-a776-7d773803feb1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.250840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 452/903] 10.222.0.95 () {56 vars in 1149 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:08.253621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.379482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360b126-02a0-4eff-a966-b1e11a6b2af8 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.382820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b57faa-700d-48da-a776-7d773803feb1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.394353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.420014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a316940-ac75-48b4-aff2-9734f8890c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.420696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a316940-ac75-48b4-aff2-9734f8890c64 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.420970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a316940-ac75-48b4-aff2-9734f8890c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.422010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a316940-ac75-48b4-aff2-9734f8890c64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.427978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.428610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.428979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.429261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.429894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:08.430191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.431655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.438166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 452/904] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:08.444016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.474251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.521218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a316940-ac75-48b4-aff2-9734f8890c64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.523762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a316940-ac75-48b4-aff2-9734f8890c64 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.559196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc9043-8a72-4bdb-9d42-7ab458dcaa33 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.596740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.597506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.597907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.598315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.599112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] RBAC: Adding query filter params (interface=public, service_id=574d92bbfd9b4c0fb58b1c2c6cc02c95, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:08.599675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.601262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.610472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 453/905] 10.222.0.95 () {56 vars in 1251 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/endpoints?service_id=574d92bbfd9b4c0fb58b1c2c6cc02c95&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:08.616004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.652935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 453/906] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:08.658978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a316940-ac75-48b4-aff2-9734f8890c64 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:08.674456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.717606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:08.730243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8f778-c8ba-4626-8bf9-f318889db4b5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.766764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68c517-3a08-4805-9f43-2e3060881063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.767543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68c517-3a08-4805-9f43-2e3060881063 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.767892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68c517-3a08-4805-9f43-2e3060881063 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.768235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68c517-3a08-4805-9f43-2e3060881063 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.769469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68c517-3a08-4805-9f43-2e3060881063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.771750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68c517-3a08-4805-9f43-2e3060881063 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.780252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a316940-ac75-48b4-aff2-9734f8890c64 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:08.782518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 454/907] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:05:08.833316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:08.834282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:08.834752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:08.835238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:08.848236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:08.850005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:08.868901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:08.871449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:08.872626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:08.872626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:08.872626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:08.874559 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 454/908] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:08.880451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68c517-3a08-4805-9f43-2e3060881063 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:09.013345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68c517-3a08-4805-9f43-2e3060881063 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:09.052021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:09.052710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.052990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.053260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.053963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:09.054266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:09.055897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:09.070950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 455/909] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:09.077632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:09.157473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:09.197524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b4cc2-bb84-43cc-8b5b-9c1313505410 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:09.236039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:09.236711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.236995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.237265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.244236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:09.246377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:09.257970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:09.259479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:09.260996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 455/910] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:09.266172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:09.283702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:09.379007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e0f31-b887-40d0-96e6-9303b92f1d56 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:09.416291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe4628-c858-433d-9448-672547b9c7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:09.417677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4628-c858-433d-9448-672547b9c7ab None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.418092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4628-c858-433d-9448-672547b9c7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.418441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe4628-c858-433d-9448-672547b9c7ab None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.419541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe4628-c858-433d-9448-672547b9c7ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:09.419859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe4628-c858-433d-9448-672547b9c7ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:09.421588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe4628-c858-433d-9448-672547b9c7ab None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:09.436999 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 456/911] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:09.442716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:09.547402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e30b81-740c-4ebf-83a5-77373c51f8d7 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.547589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e30b81-740c-4ebf-83a5-77373c51f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.547853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e30b81-740c-4ebf-83a5-77373c51f8d7 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.549802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 457/912] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:09.554585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27515576-d989-413e-9b51-f058f734cda5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.554868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27515576-d989-413e-9b51-f058f734cda5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.554981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba47f55-ba51-449e-ba5d-1a9698cc2165 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:09.555239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27515576-d989-413e-9b51-f058f734cda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.574898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-27515576-d989-413e-9b51-f058f734cda5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:09.576162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-27515576-d989-413e-9b51-f058f734cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:09.580501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:09.592916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:09.593521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.593797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.594063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.594787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] RBAC: Adding query filter params (role.id=e7d6cbfad8444cc39b36987330c71769, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=8a1dcd805548450cb3411c62083a4bc5) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:09.595096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:09.596534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:09.602587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 456/913] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:09 2022] GET /identity/v3/role_assignments?user.id=8a1dcd805548450cb3411c62083a4bc5&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=e7d6cbfad8444cc39b36987330c71769 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:09.657613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 458/914] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:09.927973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.087173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47b48-1150-42df-93c7-e8b7cb57b66f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.123663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.124300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.124594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.124867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.125449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.126822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:10.137341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:10.138386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:10.139234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 457/915] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:10.145115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27515576-d989-413e-9b51-f058f734cda5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.148866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.248372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27515576-d989-413e-9b51-f058f734cda5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.284133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14a0ff-4739-4607-9143-0fde0bb5f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.284818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14a0ff-4739-4607-9143-0fde0bb5f3bb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.285074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14a0ff-4739-4607-9143-0fde0bb5f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.285304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14a0ff-4739-4607-9143-0fde0bb5f3bb None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.285938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14a0ff-4739-4607-9143-0fde0bb5f3bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:10.286241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14a0ff-4739-4607-9143-0fde0bb5f3bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.287610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14a0ff-4739-4607-9143-0fde0bb5f3bb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.293635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 459/916] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:10.299572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.328283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833433b-ab88-41da-b284-8c815ea1e69a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.328636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833433b-ab88-41da-b284-8c815ea1e69a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.328864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833433b-ab88-41da-b284-8c815ea1e69a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.330451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 460/917] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:10.335298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fbc60-d971-4d76-8e87-0328aafe4201 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.335583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fbc60-d971-4d76-8e87-0328aafe4201 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.335862 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fbc60-d971-4d76-8e87-0328aafe4201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.355582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-976fbc60-d971-4d76-8e87-0328aafe4201 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:10.356813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-976fbc60-d971-4d76-8e87-0328aafe4201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:10.361521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.422239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed98387-4a1e-4a04-a081-68cb7a1c5872 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.426874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.441050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 461/918] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:10.459610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.460222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.460497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.460790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.461425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:10.461789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.463169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.469319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 458/919] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:10.475241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fbc60-d971-4d76-8e87-0328aafe4201 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.579236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fbc60-d971-4d76-8e87-0328aafe4201 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.614514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.615122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.615398 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.615664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.616362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] RBAC: Adding query filter params (interface=public, service_id=ae3ca752746a4d83b2348556af1898d4, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:10.616689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.618052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.623833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 462/920] 10.222.0.95 () {56 vars in 1251 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/endpoints?service_id=ae3ca752746a4d83b2348556af1898d4&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:10.629421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.726737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:10.749174 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea40b4-97f9-4a2d-9e1a-902af3f76f00 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.772124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.798522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8fc31-dc4a-4891-9394-fc6efd2d80e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.799408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8fc31-dc4a-4891-9394-fc6efd2d80e6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.799687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8fc31-dc4a-4891-9394-fc6efd2d80e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.799956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8fc31-dc4a-4891-9394-fc6efd2d80e6 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.800597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8fc31-dc4a-4891-9394-fc6efd2d80e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.800939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.802066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8fc31-dc4a-4891-9394-fc6efd2d80e6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.820602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 459/921] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:05:10.859801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946583e-35bf-492e-b708-9d8061c89bc0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.890074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198613f-ea71-44b9-86a4-5877cc273e95 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.890334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198613f-ea71-44b9-86a4-5877cc273e95 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.890696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198613f-ea71-44b9-86a4-5877cc273e95 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.892293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 460/922] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:10.898303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d934b-19d2-469c-a154-e6613fa72e50 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.898721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d934b-19d2-469c-a154-e6613fa72e50 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.899094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d934b-19d2-469c-a154-e6613fa72e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.905583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.906565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.907021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.907470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.908425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.910997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.922043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-0d1d934b-19d2-469c-a154-e6613fa72e50 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:10.923658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0d1d934b-19d2-469c-a154-e6613fa72e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:10.927443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:10.928465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:10.933148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 463/923] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:10.937748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.011244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 461/924] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:11.056034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23137d-420a-4133-893a-e78a217660a7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.081470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.094374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.095300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.095782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.096239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.097061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:11.097519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.099085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.107052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 464/925] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.112709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d934b-19d2-469c-a154-e6613fa72e50 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.224986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d934b-19d2-469c-a154-e6613fa72e50 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.236865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.262576 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.263193 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.263474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.263743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.264387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:11.264712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.266114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.271778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 462/926] 10.222.0.95 () {56 vars in 1114 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.278193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.321141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.402658 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7062e-1e63-40b6-b8c4-042cfc8b91f5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.444698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f519162-3e4f-41d8-a766-057e295b34f6 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.451606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f0b69-10c7-47ac-8c92-244e5bda3afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.452522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f0b69-10c7-47ac-8c92-244e5bda3afc None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.452975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f0b69-10c7-47ac-8c92-244e5bda3afc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.453426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f0b69-10c7-47ac-8c92-244e5bda3afc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.454891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f0b69-10c7-47ac-8c92-244e5bda3afc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.456931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f0b69-10c7-47ac-8c92-244e5bda3afc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.492410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 465/927] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:05:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:11.506845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388614c-3267-476a-aaee-b25029a8403e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.507557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388614c-3267-476a-aaee-b25029a8403e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.507845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388614c-3267-476a-aaee-b25029a8403e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.508117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388614c-3267-476a-aaee-b25029a8403e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.508748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388614c-3267-476a-aaee-b25029a8403e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.510220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388614c-3267-476a-aaee-b25029a8403e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f388614c-3267-476a-aaee-b25029a8403e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:11.521254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:11.522320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:05:11.523247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:11.527742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 463/928] 10.222.0.95 () {56 vars in 1132 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:11.529428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f0b69-10c7-47ac-8c92-244e5bda3afc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.586101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.662377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f0b69-10c7-47ac-8c92-244e5bda3afc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.694469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5806-709f-44d4-8b04-ddfc2b176202 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.694665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5806-709f-44d4-8b04-ddfc2b176202 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.694929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d5806-709f-44d4-8b04-ddfc2b176202 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.696505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 464/929] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:11.703708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb96fab-e61b-4a54-805a-f0d847440e10 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.703987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb96fab-e61b-4a54-805a-f0d847440e10 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.704251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb96fab-e61b-4a54-805a-f0d847440e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.705948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.706912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.707445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.707867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.708788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:11.709094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.710788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.717733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 466/930] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:11.723378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:11.731021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-efb96fab-e61b-4a54-805a-f0d847440e10 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:11.731021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-efb96fab-e61b-4a54-805a-f0d847440e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:11.778067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.844629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 465/931] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:11.857279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c8fd2-6085-4313-9688-630f5ff75d1d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:11.867109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:11.895370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:11.895991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:11.896493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:11.897091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:11.897733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:11.898525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:11.899971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:11.906356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 467/932] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:11.918674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb96fab-e61b-4a54-805a-f0d847440e10 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.037151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb96fab-e61b-4a54-805a-f0d847440e10 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.078066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52ad9d-346c-4ca1-89b1-728d9768744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.078699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ad9d-346c-4ca1-89b1-728d9768744e None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.078979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ad9d-346c-4ca1-89b1-728d9768744e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.079252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52ad9d-346c-4ca1-89b1-728d9768744e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.079951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52ad9d-346c-4ca1-89b1-728d9768744e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.081431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52ad9d-346c-4ca1-89b1-728d9768744e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.105373 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 466/933] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:05:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:12.139971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.262857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e108dc-f2f2-4f53-971b-4ccdaa3f84d5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.305110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413018b-8794-4749-90d1-18b7cb9a7a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.305742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413018b-8794-4749-90d1-18b7cb9a7a1f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.306028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413018b-8794-4749-90d1-18b7cb9a7a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.306286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413018b-8794-4749-90d1-18b7cb9a7a1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.319451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413018b-8794-4749-90d1-18b7cb9a7a1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3413018b-8794-4749-90d1-18b7cb9a7a1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:12.329573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:12.330582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:12.331439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:12.332882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 468/934] 10.222.0.95 () {56 vars in 1120 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:12.337561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52ad9d-346c-4ca1-89b1-728d9768744e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.361747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.452314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52ad9d-346c-4ca1-89b1-728d9768744e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.489154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.490170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.490663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.491137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.492034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:12.492551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.494167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.504314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 467/935] 10.222.0.95 () {56 vars in 1115 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:12.510284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413018b-8794-4749-90d1-18b7cb9a7a1f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.540964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.624975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413018b-8794-4749-90d1-18b7cb9a7a1f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.661061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.661705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.661987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.662309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.674889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.676367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:12.694490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:12.695506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:12.696984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:12.696984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:12.696984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:12.696984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 469/936] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:12.702006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.789937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.812073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a166e-54d4-4d90-8b8d-5b76087b0fc0 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:12.847056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:12.847675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.847954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.848223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.848981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:12.849287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:12.850692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:12.865852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 468/937] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:12.872201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.958745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.980259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce715-bb27-4c45-b50f-83cd108bb819 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.015826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e026bc-ae82-4ce6-aff0-cff48bd1540f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.016463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e026bc-ae82-4ce6-aff0-cff48bd1540f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.016776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e026bc-ae82-4ce6-aff0-cff48bd1540f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.017050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e026bc-ae82-4ce6-aff0-cff48bd1540f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.025297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e026bc-ae82-4ce6-aff0-cff48bd1540f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.027317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e026bc-ae82-4ce6-aff0-cff48bd1540f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b1e026bc-ae82-4ce6-aff0-cff48bd1540f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:13.039438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:13.040445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:13.042815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 470/938] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:13.047208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.071284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001a27e-88fd-447f-bdb8-39bb8117f686 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.071567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001a27e-88fd-447f-bdb8-39bb8117f686 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.071748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001a27e-88fd-447f-bdb8-39bb8117f686 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.073464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 471/939] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:13.079446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0a653-d27c-4da5-b682-2c68022faf75 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.080024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0a653-d27c-4da5-b682-2c68022faf75 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.080497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0a653-d27c-4da5-b682-2c68022faf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.101771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-59d0a653-d27c-4da5-b682-2c68022faf75 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:13.103353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-59d0a653-d27c-4da5-b682-2c68022faf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:13.165383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd033f-98a5-4d19-a323-e8c7284083e5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.188145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 472/940] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:13.204209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.204857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.205137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.205412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.206148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:13.206453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.207909 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.223360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.232890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 469/941] 10.222.0.95 () {56 vars in 1149 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:13.234478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0a653-d27c-4da5-b682-2c68022faf75 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.312416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.348784 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0a653-d27c-4da5-b682-2c68022faf75 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.390362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.391009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.391292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.391565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.403374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.404869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:13.421242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:13.422243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:13.423128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:13.423128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:13.423128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:13.423128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 473/942] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:13.430222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.520608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.547978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cae56-6da0-4544-902f-cce7f3be21c4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.585903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d0bd4-8a33-437b-8438-e4691f26cd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.586822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d0bd4-8a33-437b-8438-e4691f26cd35 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.586988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d0bd4-8a33-437b-8438-e4691f26cd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.587382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d0bd4-8a33-437b-8438-e4691f26cd35 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.587938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d0bd4-8a33-437b-8438-e4691f26cd35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:13.588382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d0bd4-8a33-437b-8438-e4691f26cd35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.590961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d0bd4-8a33-437b-8438-e4691f26cd35 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.604828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 470/943] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:13.606634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be36a0-2156-4c62-a8cf-3e7d13411c31 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.606933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be36a0-2156-4c62-a8cf-3e7d13411c31 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.607199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be36a0-2156-4c62-a8cf-3e7d13411c31 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.608475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 471/944] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:13.611809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be36a0-2156-4c62-a8cf-3e7d13411c31 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.638878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a6d73-372c-48cc-b75d-2d4321ff939c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.664197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.676816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15ff0a-d207-4962-8b20-f9e81e02664a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.677447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ff0a-d207-4962-8b20-f9e81e02664a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.677864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ff0a-d207-4962-8b20-f9e81e02664a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.678145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ff0a-d207-4962-8b20-f9e81e02664a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.678733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15ff0a-d207-4962-8b20-f9e81e02664a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.680102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15ff0a-d207-4962-8b20-f9e81e02664a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.683490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-cf15ff0a-d207-4962-8b20-f9e81e02664a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:13.689487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:13.690515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:13.691360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 474/945] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:13.692163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db6aac-6a5f-42fc-b776-6c3957138282 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.693082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db6aac-6a5f-42fc-b776-6c3957138282 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.693082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db6aac-6a5f-42fc-b776-6c3957138282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.712866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f0db6aac-6a5f-42fc-b776-6c3957138282 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:13.715461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f0db6aac-6a5f-42fc-b776-6c3957138282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:13.739131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be36a0-2156-4c62-a8cf-3e7d13411c31 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.777900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.779054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.779054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.779054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.786626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.788229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.796601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 475/946] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:13.796601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db6aac-6a5f-42fc-b776-6c3957138282 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:13.801084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:13.802005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:13.802879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:13.802879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:13.802879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:13.802879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:13.802879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:13.802879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:13.802879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:13.802879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:13.807556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 472/947] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:13.808853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:13.927967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21752f7-eaa9-4beb-9cdb-a5be3fc134d5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.938663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db6aac-6a5f-42fc-b776-6c3957138282 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.953779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.962908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.971255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.971874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.972154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.972427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.973199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:13.973488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.974941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.989793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.990681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.991102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.991527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.992303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:13.992788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.994098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 473/948] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:13.994579 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.995161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.001690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 476/949] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:14.008618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.108493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952b12b-3dc0-49b1-a3c8-c463401de06a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.134111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4bc78-a482-440e-b596-dbba25aa83d5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.160206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e978c-21a4-4566-aecc-55f9516ee2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.160919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e978c-21a4-4566-aecc-55f9516ee2e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.161218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e978c-21a4-4566-aecc-55f9516ee2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.161491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e978c-21a4-4566-aecc-55f9516ee2e2 None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/8a1dcd805548450cb3411c62083a4bc5/roles/e7d6cbfad8444cc39b36987330c71769` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.164129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e978c-21a4-4566-aecc-55f9516ee2e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=8a1dcd805548450cb3411c62083a4bc5, role_id=e7d6cbfad8444cc39b36987330c71769)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.165633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e978c-21a4-4566-aecc-55f9516ee2e2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.180493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.181447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 474/950] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:13 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/8a1dcd805548450cb3411c62083a4bc5/roles/e7d6cbfad8444cc39b36987330c71769 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:14.181680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.182709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.182899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e978c-21a4-4566-aecc-55f9516ee2e2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.183376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.184187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:14.184490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.185944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.200620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 477/951] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:14.204100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.249943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.270128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.332466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e978c-21a4-4566-aecc-55f9516ee2e2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.341250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23411-ec98-44ac-8e03-e7e3842a3728 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.380914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.381717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.382002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.382172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8aa3b-6437-4b20-bb08-f53334f345b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.382346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.382767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8aa3b-6437-4b20-bb08-f53334f345b7 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.382994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.383107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8aa3b-6437-4b20-bb08-f53334f345b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.383310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8aa3b-6437-4b20-bb08-f53334f345b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.383987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8aa3b-6437-4b20-bb08-f53334f345b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.384466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.386141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8aa3b-6437-4b20-bb08-f53334f345b7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.389118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING keystone.server.flask.common [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 19:05:14.409937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.sql.core [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:05:14.409937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 19:05:14.409937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [parameters: {'id': 'eac5c46ce73c40978e759ccc6249579b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 19:05:14.409937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81592) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 19:05:14.424252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 478/952] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:05:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eac5c46ce73c40978e759ccc6249579b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eac5c46ce73c40978e759ccc6249579b. Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:05:14.424463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application result.read() Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 19:05:14.425305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application yield resource Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise self.value Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application session.commit() Aug 31 19:05:14.426287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.session.flush() Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application compat.raise_( Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 19:05:14.427227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 19:05:14.428142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application result.read() Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:05:14.428960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [parameters: {'id': 'eac5c46ce73c40978e759ccc6249579b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:14.429979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:14.431524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eac5c46ce73c40978e759ccc6249579b. Aug 31 19:05:14.437325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:14.438032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f741b-99e0-4a60-b616-d17a3844972c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.438032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f741b-99e0-4a60-b616-d17a3844972c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.438032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f741b-99e0-4a60-b616-d17a3844972c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.438237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 475/953] 10.222.0.95 () {60 vars in 1140 bytes} [Wed Aug 31 19:05:14 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:14.438315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 479/954] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:14.438315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dc8e7-08cb-49fb-be2d-8874a30c6657 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.438445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.438516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dc8e7-08cb-49fb-be2d-8874a30c6657 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.438516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55dc8e7-08cb-49fb-be2d-8874a30c6657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.461154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b55dc8e7-08cb-49fb-be2d-8874a30c6657 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:14.462343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b55dc8e7-08cb-49fb-be2d-8874a30c6657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:14.591214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 480/955] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:14.597777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.604954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55c1ac-1868-460a-8faa-5f6e200accf7 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.658888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.659542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.659824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.660098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.670932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.673435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:14.692776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:14.693797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:14.695516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 476/956] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:14.701731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dc8e7-08cb-49fb-be2d-8874a30c6657 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.737840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:14.839205 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55dc8e7-08cb-49fb-be2d-8874a30c6657 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:14.876344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:14.896544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:14.897196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:14.897472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:14.897735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:14.898413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:14.898708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:14.900153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:14.920009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 481/957] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:05:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:15.037923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01eb6e-be63-43d1-b622-c276aa684c33 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:15.088521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:15.089358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.089642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.090119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.090746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:15.092779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:15.103375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:15.105120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:15.107061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 477/958] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:15.112629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:15.247376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:15.268915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa8ae5-bb6b-48f4-8e84-c795552eb818 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:15.313292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fc455-460c-48ff-9349-e00db90d8da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:15.314189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fc455-460c-48ff-9349-e00db90d8da4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.314698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fc455-460c-48ff-9349-e00db90d8da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.315119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fc455-460c-48ff-9349-e00db90d8da4 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.315906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fc455-460c-48ff-9349-e00db90d8da4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:15.316370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fc455-460c-48ff-9349-e00db90d8da4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:15.317926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fc455-460c-48ff-9349-e00db90d8da4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:15.324082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 482/959] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:05:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:15.331441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:15.372281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:15.466971 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1c9aa-e82f-4b5f-a1c4-f5f3fb15d28f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:15.504156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e6c56-cf8d-41a6-beac-d5c0d99775e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:15.504865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6c56-cf8d-41a6-beac-d5c0d99775e4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.505147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6c56-cf8d-41a6-beac-d5c0d99775e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.505422 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6c56-cf8d-41a6-beac-d5c0d99775e4 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.506062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e6c56-cf8d-41a6-beac-d5c0d99775e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:15.506366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e6c56-cf8d-41a6-beac-d5c0d99775e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:15.507775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e6c56-cf8d-41a6-beac-d5c0d99775e4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:15.516880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 478/960] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:05:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:16.299549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c71c21-a2ef-4f24-80b3-728416710e3c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.300108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c71c21-a2ef-4f24-80b3-728416710e3c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.300108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c71c21-a2ef-4f24-80b3-728416710e3c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.301705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 483/961] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:16.307255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ebf2-45e7-4117-a81f-83a2f83c213b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.307538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ebf2-45e7-4117-a81f-83a2f83c213b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.307812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489ebf2-45e7-4117-a81f-83a2f83c213b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.326920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d489ebf2-45e7-4117-a81f-83a2f83c213b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:16.328426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d489ebf2-45e7-4117-a81f-83a2f83c213b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:16.403822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 479/962] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:16.645637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcefeb-1c4a-4406-8c79-73165d43899d None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.646017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcefeb-1c4a-4406-8c79-73165d43899d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.646302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcefeb-1c4a-4406-8c79-73165d43899d None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.647988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 484/963] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:16.654278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15585d-c828-4839-9456-2326f99f2fe3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.654756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15585d-c828-4839-9456-2326f99f2fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.654756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15585d-c828-4839-9456-2326f99f2fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.675349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-2b15585d-c828-4839-9456-2326f99f2fe3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:16.676628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2b15585d-c828-4839-9456-2326f99f2fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:16.685973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcefeb-1c4a-4406-8c79-73165d43899d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:16.687698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:16.776727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 480/964] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:16.815916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcefeb-1c4a-4406-8c79-73165d43899d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:16.853200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1244ad7-8eda-4254-9106-0c1880c0c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:16.854345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1244ad7-8eda-4254-9106-0c1880c0c394 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.854829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1244ad7-8eda-4254-9106-0c1880c0c394 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.855325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1244ad7-8eda-4254-9106-0c1880c0c394 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.866334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1244ad7-8eda-4254-9106-0c1880c0c394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d1244ad7-8eda-4254-9106-0c1880c0c394 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:16.871975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:16.872935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:16.873814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:16.876447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 485/965] 10.222.0.95 () {56 vars in 1120 bytes} [Wed Aug 31 19:05:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:16.882564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15585d-c828-4839-9456-2326f99f2fe3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:16.991651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15585d-c828-4839-9456-2326f99f2fe3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.030000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.030990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.031478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.031967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.032891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:17.033410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.035007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.049212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 481/966] 10.222.0.95 () {56 vars in 1115 bytes} [Wed Aug 31 19:05:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:17.051228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1244ad7-8eda-4254-9106-0c1880c0c394 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.062414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.068475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.075510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.171115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1244ad7-8eda-4254-9106-0c1880c0c394 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.190059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e37f61-d319-42fc-ae12-90d2ca97e02a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.209407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00833e7-f605-4e1a-badc-ee434f228c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.210204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00833e7-f605-4e1a-badc-ee434f228c20 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.210665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00833e7-f605-4e1a-badc-ee434f228c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.211080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00833e7-f605-4e1a-badc-ee434f228c20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.223497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00833e7-f605-4e1a-badc-ee434f228c20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.224989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00833e7-f605-4e1a-badc-ee434f228c20 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.231347 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.232115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.232399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.232701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.233296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.234823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b00833e7-f605-4e1a-badc-ee434f228c20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:17.241234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:17.242252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:17.243149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:17.243149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:17.243149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:17.244153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:17.245142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:17.245985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:17.247830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 486/967] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:17.249070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c21490-b079-4731-a144-a938fa1ed900 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.249358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c21490-b079-4731-a144-a938fa1ed900 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.249637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c21490-b079-4731-a144-a938fa1ed900 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.250823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 482/968] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:17.252743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 487/969] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:17.252840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.253798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c21490-b079-4731-a144-a938fa1ed900 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.318371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.338088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.365356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c21490-b079-4731-a144-a938fa1ed900 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.376134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19729c1-4a02-46fc-b37d-494ebb86757a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.407360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73396081-a338-4cf5-8c41-df012530c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.407981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73396081-a338-4cf5-8c41-df012530c082 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.408261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73396081-a338-4cf5-8c41-df012530c082 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.408537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73396081-a338-4cf5-8c41-df012530c082 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.409201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73396081-a338-4cf5-8c41-df012530c082 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:17.409507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73396081-a338-4cf5-8c41-df012530c082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.410885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73396081-a338-4cf5-8c41-df012530c082 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.419701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.420325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.420710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.420906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.421430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 488/970] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:17.421538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:17.421842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.422744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670d86d-00fb-4387-b706-51967baef8b4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.423042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670d86d-00fb-4387-b706-51967baef8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.423253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.423349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670d86d-00fb-4387-b706-51967baef8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.437500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 483/971] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:17.438355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.447607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5670d86d-00fb-4387-b706-51967baef8b4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:17.448916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5670d86d-00fb-4387-b706-51967baef8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:17.536701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 489/972] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:17.536701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670d86d-00fb-4387-b706-51967baef8b4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.560454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73335aa-b076-4088-9707-228b2dc53d9a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.603747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.607138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.616018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf406f8-0d6f-49a7-b605-5faadac04cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.616824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf406f8-0d6f-49a7-b605-5faadac04cee None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.617278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf406f8-0d6f-49a7-b605-5faadac04cee None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.617716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf406f8-0d6f-49a7-b605-5faadac04cee None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.618998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf406f8-0d6f-49a7-b605-5faadac04cee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:17.619257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf406f8-0d6f-49a7-b605-5faadac04cee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.622310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf406f8-0d6f-49a7-b605-5faadac04cee None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.629355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 484/973] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:17.634712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe136d-3a6c-4ee6-a2e0-55f2af1c4b68 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.635030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe136d-3a6c-4ee6-a2e0-55f2af1c4b68 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.635495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe136d-3a6c-4ee6-a2e0-55f2af1c4b68 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.638044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 485/974] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:17.638840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe136d-3a6c-4ee6-a2e0-55f2af1c4b68 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.661935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670d86d-00fb-4387-b706-51967baef8b4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.702176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a1508-4d06-491e-8b00-3e65268ce521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.702809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a1508-4d06-491e-8b00-3e65268ce521 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.703091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a1508-4d06-491e-8b00-3e65268ce521 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.703361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a1508-4d06-491e-8b00-3e65268ce521 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.711769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a1508-4d06-491e-8b00-3e65268ce521 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.713298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a1508-4d06-491e-8b00-3e65268ce521 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-7a4a1508-4d06-491e-8b00-3e65268ce521 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:17.727050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:17.728108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:17.729433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 490/975] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:17.730332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728caa2-b1ce-4ffa-b6c8-ea5372976d31 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.731111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728caa2-b1ce-4ffa-b6c8-ea5372976d31 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.731293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728caa2-b1ce-4ffa-b6c8-ea5372976d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.752927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e728caa2-b1ce-4ffa-b6c8-ea5372976d31 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:17.754161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e728caa2-b1ce-4ffa-b6c8-ea5372976d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:17.761395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe136d-3a6c-4ee6-a2e0-55f2af1c4b68 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.798481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.799277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.799557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.799830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.800552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] RBAC: Adding query filter params (interface=admin, service_id=ae3ca752746a4d83b2348556af1898d4, region_id=RegionOne) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:17.800889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.802388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.808504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 486/976] 10.222.0.95 () {56 vars in 1249 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/endpoints?service_id=ae3ca752746a4d83b2348556af1898d4&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:17.809372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.832071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 491/977] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:17.833329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728caa2-b1ce-4ffa-b6c8-ea5372976d31 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.877195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.887432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.928055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a11381-bbcf-4094-ad08-c39b32d9ac95 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.950738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728caa2-b1ce-4ffa-b6c8-ea5372976d31 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:17.965952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.966570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.967190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.967190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.967857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:17.968173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.969668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.989547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:17.990170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.990456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.990693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 487/978] 10.222.0.95 () {56 vars in 1149 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:17.996366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.996366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:17.996560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:17.996689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:17.996689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING keystone.server.flask.common [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 19:05:18.009771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.sql.core [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:05:18.009771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 19:05:18.009771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [parameters: {'user_id': '9933e2eebe0248a38a79c9624fa8aeca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 19:05:18.009771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81591) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result.read() Aug 31 19:05:18.026227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:05:18.027276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application yield resource Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise self.value Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application session.commit() Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 19:05:18.028299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.session.flush() Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application compat.raise_( Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 19:05:18.030363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:05:18.031360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result.read() Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:05:18.032218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [parameters: {'user_id': '9933e2eebe0248a38a79c9624fa8aeca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:18.034357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 19:05:18.036255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 19:05:18.038792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 19:05:18.038792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 19:05:18.038792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 19:05:18.038792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 19:05:18.038792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 19:05:18.038792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:18.038792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 492/979] 10.222.0.95 () {60 vars in 1131 bytes} [Wed Aug 31 19:05:17 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:18.038792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.132168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860949-eff4-4484-9f13-e740e7858ba9 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.150986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.157781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b34b8-5f07-45dc-8d55-46cf64b8a21e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.169253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.171487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cee1f-d571-4bd1-8acc-063f57dca253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.172114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cee1f-d571-4bd1-8acc-063f57dca253 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.172392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cee1f-d571-4bd1-8acc-063f57dca253 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.172702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cee1f-d571-4bd1-8acc-063f57dca253 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.173342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cee1f-d571-4bd1-8acc-063f57dca253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.175136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cee1f-d571-4bd1-8acc-063f57dca253 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.181894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 488/980] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:05:17 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:05:18.183589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cee1f-d571-4bd1-8acc-063f57dca253 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.205919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046a5e-6004-4e4a-911e-5b5efa94db97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.206668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046a5e-6004-4e4a-911e-5b5efa94db97 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.207002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046a5e-6004-4e4a-911e-5b5efa94db97 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.207336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046a5e-6004-4e4a-911e-5b5efa94db97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.296520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb046a5e-6004-4e4a-911e-5b5efa94db97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.298371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb046a5e-6004-4e4a-911e-5b5efa94db97 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.318813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cee1f-d571-4bd1-8acc-063f57dca253 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.367182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.367849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.368126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.368395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.376949 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.378593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:18.392365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:18.393536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:05:18.394554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:18.396646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 489/981] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:18.397565 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.434533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 493/982] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:18.436131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046a5e-6004-4e4a-911e-5b5efa94db97 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.448982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.461305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.545487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe82c0-2624-4947-961f-89c8694ef1a5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.558494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046a5e-6004-4e4a-911e-5b5efa94db97 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.591306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5faebe4-1222-4079-ade0-7aaea7a1d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.592244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5faebe4-1222-4079-ade0-7aaea7a1d292 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.592244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5faebe4-1222-4079-ade0-7aaea7a1d292 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.592615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5faebe4-1222-4079-ade0-7aaea7a1d292 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.593114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5faebe4-1222-4079-ade0-7aaea7a1d292 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.594534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5faebe4-1222-4079-ade0-7aaea7a1d292 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.609101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.609829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.610174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.610520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.611294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:18.611662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.616766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 490/983] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:05:18 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:18.616956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.617041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5faebe4-1222-4079-ade0-7aaea7a1d292 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.660319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 494/984] 10.222.0.95 () {56 vars in 1101 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:18.759244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.773035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5faebe4-1222-4079-ade0-7aaea7a1d292 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.817295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.818223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.818655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.818928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.832868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.834404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:18.852374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:18.853521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:18.855095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:18.855095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:18.855095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:18.855095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 491/985] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:18.860137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.933696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.973049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f3518-4f98-439e-a553-3d0d1ac1be0f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.008248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.008887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.009171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.009445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.010130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:19.010431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.011806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.023553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 495/986] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:19.029757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.148315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193dbe5-26a8-4581-a8ec-7a8a6b735dc9 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.184392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.184973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.185253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.185524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.192172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.193664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:19.205575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:19.206487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:19.207379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:19.207379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:19.207379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:19.207379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:19.207379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:19.207379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:19.207379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:19.207379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:19.208645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 492/987] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:19.212664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.320013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c79232-48b0-4aec-a64b-9b74b50032b1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.347368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.358822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9778e-f02f-44a8-ba6c-b16393fa8470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.359453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9778e-f02f-44a8-ba6c-b16393fa8470 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.359735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9778e-f02f-44a8-ba6c-b16393fa8470 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.360008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9778e-f02f-44a8-ba6c-b16393fa8470 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.360757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9778e-f02f-44a8-ba6c-b16393fa8470 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:19.361067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9778e-f02f-44a8-ba6c-b16393fa8470 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.362674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9778e-f02f-44a8-ba6c-b16393fa8470 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.380605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 496/988] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:19.382217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.394812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.493151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1b0d4-085a-429d-9532-cce666cb12c7 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.529183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b0a2d-9282-4f65-8d93-c2651b64d0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.529790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b0a2d-9282-4f65-8d93-c2651b64d0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.530071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b0a2d-9282-4f65-8d93-c2651b64d0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.530344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b0a2d-9282-4f65-8d93-c2651b64d0e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.531069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b0a2d-9282-4f65-8d93-c2651b64d0e8 None admin] RBAC: Adding query filter params (role.id=e7d6cbfad8444cc39b36987330c71769, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=8a1dcd805548450cb3411c62083a4bc5) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:19.531378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b0a2d-9282-4f65-8d93-c2651b64d0e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.532853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b0a2d-9282-4f65-8d93-c2651b64d0e8 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.544603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 493/989] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/role_assignments?user.id=8a1dcd805548450cb3411c62083a4bc5&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=e7d6cbfad8444cc39b36987330c71769 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:20.222344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b33cb9-164e-40e9-a5c7-8fc28d9b4836 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.222869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b33cb9-164e-40e9-a5c7-8fc28d9b4836 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.222869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b33cb9-164e-40e9-a5c7-8fc28d9b4836 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.224383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 497/990] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:20.229542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edefb77-0124-4087-9f3c-2711453b85df None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.235154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edefb77-0124-4087-9f3c-2711453b85df None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.235154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edefb77-0124-4087-9f3c-2711453b85df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.249210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4edefb77-0124-4087-9f3c-2711453b85df None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:20.250421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4edefb77-0124-4087-9f3c-2711453b85df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:20.323923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 494/991] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:20.610079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b33cb9-164e-40e9-a5c7-8fc28d9b4836 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.630632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef9126-8c6d-46a5-a1a8-43d0aac3dcfa None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.631036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef9126-8c6d-46a5-a1a8-43d0aac3dcfa None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.631209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef9126-8c6d-46a5-a1a8-43d0aac3dcfa None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.639776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 495/992] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:20.639776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518a887-467c-46ad-908a-9378231cc0cb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.639776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518a887-467c-46ad-908a-9378231cc0cb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.639776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518a887-467c-46ad-908a-9378231cc0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.650637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:20.665955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8518a887-467c-46ad-908a-9378231cc0cb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:20.667253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8518a887-467c-46ad-908a-9378231cc0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:20.737562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b33cb9-164e-40e9-a5c7-8fc28d9b4836 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.743741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 496/993] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:20.773088 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.773724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.774006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.774279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.774866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.776353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:20.785901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:20.786919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:05:20.787764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:20.792119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 498/994] 10.222.0.95 () {56 vars in 1126 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:20.793111 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518a887-467c-46ad-908a-9378231cc0cb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:20.908553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518a887-467c-46ad-908a-9378231cc0cb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:20.940868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:20.947649 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:20.948765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:20.949296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:20.949785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:20.950638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:20.951157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:20.952776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:20.967696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 497/995] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:20.969886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.023047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.031085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.086301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c80f9c-80fb-4e22-b44c-8359ab6ca544 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.128016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.128745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.129352 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.129631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.130571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:21.130878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.132844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.140732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 499/996] 10.222.0.95 () {56 vars in 1122 bytes} [Wed Aug 31 19:05:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:21.145197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.156213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b29d2-8c63-4d6b-8d30-a35dbbfd3e0c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.192546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.193116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.193395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.193662 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] PATH_INFO: `/v3/services/compute` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.194244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.195617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:21.204544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:21.205643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:21.206495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 498/997] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:21.211503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.267183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b3061-1bc8-4155-84e2-fbdbb8dcb419 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.275215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.284164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.307667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c4094-e349-497d-8426-0435d17e2e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.308275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c4094-e349-497d-8426-0435d17e2e84 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.308551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c4094-e349-497d-8426-0435d17e2e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.308845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c4094-e349-497d-8426-0435d17e2e84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.309512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c4094-e349-497d-8426-0435d17e2e84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.310879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c4094-e349-497d-8426-0435d17e2e84 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.323736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ea816-19da-4f5d-9565-4500b10429ce None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.328633 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 500/998] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:05:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:21.381108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114ef86-27d2-47e8-8f05-bc9938074a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.381745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ef86-27d2-47e8-8f05-bc9938074a29 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.382022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ef86-27d2-47e8-8f05-bc9938074a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.382288 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114ef86-27d2-47e8-8f05-bc9938074a29 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.382939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114ef86-27d2-47e8-8f05-bc9938074a29 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:21.383246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114ef86-27d2-47e8-8f05-bc9938074a29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.384680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114ef86-27d2-47e8-8f05-bc9938074a29 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.396810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 499/999] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:21.397966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c4094-e349-497d-8426-0435d17e2e84 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.539619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c4094-e349-497d-8426-0435d17e2e84 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.542876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b7ba-d916-4ffe-b9a9-8e2a14bbd890 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.543168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b7ba-d916-4ffe-b9a9-8e2a14bbd890 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.543443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b7ba-d916-4ffe-b9a9-8e2a14bbd890 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.544847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 500/1000] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:21.550653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30b8c-251c-41ff-af5f-c05a513900a9 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.551007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30b8c-251c-41ff-af5f-c05a513900a9 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.551291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd30b8c-251c-41ff-af5f-c05a513900a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.570908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-dcd30b8c-251c-41ff-af5f-c05a513900a9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:21.572149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-dcd30b8c-251c-41ff-af5f-c05a513900a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:21.590462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.591097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.591771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.591771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.592286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:21.592598 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.594000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.608154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 501/1001] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:21.609460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.659202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.663785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 501/1002] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:21.742110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe7877-8472-4dfd-a33e-0cca5bd73944 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.784274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.784923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.785203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.785474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.786241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] RBAC: Adding query filter params (interface=public, service_id=5c01e40aeb7b46f9a3e4c4c98fb9f822, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:21.786551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.787930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.798013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 502/1003] 10.222.0.95 () {56 vars in 1251 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/endpoints?service_id=5c01e40aeb7b46f9a3e4c4c98fb9f822&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:21.799932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd30b8c-251c-41ff-af5f-c05a513900a9 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.854955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.906886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd30b8c-251c-41ff-af5f-c05a513900a9 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:21.943593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981dacc-b26c-4cf7-9c1e-4e0c0f2a0f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:21.944197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981dacc-b26c-4cf7-9c1e-4e0c0f2a0f7b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.944481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981dacc-b26c-4cf7-9c1e-4e0c0f2a0f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.944774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981dacc-b26c-4cf7-9c1e-4e0c0f2a0f7b None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.945565 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981dacc-b26c-4cf7-9c1e-4e0c0f2a0f7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:21.946948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981dacc-b26c-4cf7-9c1e-4e0c0f2a0f7b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:21.947558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.954114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 502/1004] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:05:22.108477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c715635-c574-42eb-a68f-fb2e68d51911 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:22.110558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:22.148866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:22.149501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.149814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.150094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] PATH_INFO: `/v3/services/image` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.151271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:22.152921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:22.161990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:22.163017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:22.163888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 503/1005] 10.222.0.95 () {56 vars in 1105 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/services/image => generated 86 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:22.169894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981dacc-b26c-4cf7-9c1e-4e0c0f2a0f7b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:22.277818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981dacc-b26c-4cf7-9c1e-4e0c0f2a0f7b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:22.293904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:22.320314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d281d-dc24-4482-8ad1-ca718ec60ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:22.320973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d281d-dc24-4482-8ad1-ca718ec60ccf None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.321252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d281d-dc24-4482-8ad1-ca718ec60ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.321524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d281d-dc24-4482-8ad1-ca718ec60ccf None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.322163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d281d-dc24-4482-8ad1-ca718ec60ccf None admin] RBAC: Adding query filter params (name=image) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:22.324698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d281d-dc24-4482-8ad1-ca718ec60ccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:22.326844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d281d-dc24-4482-8ad1-ca718ec60ccf None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:22.332701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 503/1006] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:22.338268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:22.466154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360777b-7d07-4718-942b-1d96942dc36b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:22.504267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a1d2c-70ef-4626-8ab5-1a96b65bc00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:22.504902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1d2c-70ef-4626-8ab5-1a96b65bc00e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.505182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1d2c-70ef-4626-8ab5-1a96b65bc00e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.505454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a1d2c-70ef-4626-8ab5-1a96b65bc00e None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.506080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a1d2c-70ef-4626-8ab5-1a96b65bc00e None admin] RBAC: Adding query filter params (type=image) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:22.506383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a1d2c-70ef-4626-8ab5-1a96b65bc00e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:22.507818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a1d2c-70ef-4626-8ab5-1a96b65bc00e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:22.516611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 504/1007] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:23.305548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34234ea-de60-4fb4-accb-98e7380541aa None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.306107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34234ea-de60-4fb4-accb-98e7380541aa None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.306107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34234ea-de60-4fb4-accb-98e7380541aa None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.307829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 504/1008] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:23.313024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1bb3d-742a-4db5-8051-988b9ce42cbc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.313321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1bb3d-742a-4db5-8051-988b9ce42cbc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.313599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1bb3d-742a-4db5-8051-988b9ce42cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.333181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-22a1bb3d-742a-4db5-8051-988b9ce42cbc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:23.334723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-22a1bb3d-742a-4db5-8051-988b9ce42cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:23.360621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:23.409084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 505/1009] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:23.682902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34234ea-de60-4fb4-accb-98e7380541aa None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:23.798365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34234ea-de60-4fb4-accb-98e7380541aa None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:23.840506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a28f4-ecf2-4b21-9642-129ff8624982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:23.841167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a28f4-ecf2-4b21-9642-129ff8624982 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.841450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a28f4-ecf2-4b21-9642-129ff8624982 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.841725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a28f4-ecf2-4b21-9642-129ff8624982 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.842850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a28f4-ecf2-4b21-9642-129ff8624982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:23.844298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a28f4-ecf2-4b21-9642-129ff8624982 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:23.846107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 505/1010] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:23.851670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1bb3d-742a-4db5-8051-988b9ce42cbc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:23.918911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e9e43-ced2-4a4a-bbfd-1e5877b97c8b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.919260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e9e43-ced2-4a4a-bbfd-1e5877b97c8b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.919468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e9e43-ced2-4a4a-bbfd-1e5877b97c8b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.921097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 506/1011] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:23.925930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68219-3fe9-4797-9e8b-31783f13f2f3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:23.926211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68219-3fe9-4797-9e8b-31783f13f2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:23.926480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68219-3fe9-4797-9e8b-31783f13f2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:23.946501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-45e68219-3fe9-4797-9e8b-31783f13f2f3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:23.948175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-45e68219-3fe9-4797-9e8b-31783f13f2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:23.974305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1bb3d-742a-4db5-8051-988b9ce42cbc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:23.999508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.015141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284cac0-3aae-42a8-975e-21a915382b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.015748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284cac0-3aae-42a8-975e-21a915382b94 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.016046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284cac0-3aae-42a8-975e-21a915382b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.016560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284cac0-3aae-42a8-975e-21a915382b94 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.016964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284cac0-3aae-42a8-975e-21a915382b94 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.018413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284cac0-3aae-42a8-975e-21a915382b94 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.043687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 507/1012] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:24.053331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 506/1013] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:05:23 2022] POST /identity/v3/projects => generated 311 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:24.311775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68219-3fe9-4797-9e8b-31783f13f2f3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.428297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68219-3fe9-4797-9e8b-31783f13f2f3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.464129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.464804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.465073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.465177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49d43c-061c-4879-a869-7035e00d6689 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.465602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.465681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49d43c-061c-4879-a869-7035e00d6689 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.465681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49d43c-061c-4879-a869-7035e00d6689 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.465958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.467144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 507/1014] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:24.467313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.472149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2b5c-0a25-4eb3-a0ed-8a948ac0b245 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.472434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2b5c-0a25-4eb3-a0ed-8a948ac0b245 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.472745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d2b5c-0a25-4eb3-a0ed-8a948ac0b245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:24.476592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:24.477544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:24.479370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 508/1015] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:24.483981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.495426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-555d2b5c-0a25-4eb3-a0ed-8a948ac0b245 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:24.496787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-555d2b5c-0a25-4eb3-a0ed-8a948ac0b245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:24.504976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.531356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.576736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 508/1016] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:24.597802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279d2e9-e649-42e3-8545-62b57794369c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.634529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.635837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.636137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.636511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.637143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:24.637443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.638865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.644062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 509/1017] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:05:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:24.649538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d2b5c-0a25-4eb3-a0ed-8a948ac0b245 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.758903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d2b5c-0a25-4eb3-a0ed-8a948ac0b245 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.794243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.794914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.795199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.795471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.796096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:24.796400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.802034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.807776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 509/1018] 10.222.0.95 () {56 vars in 1112 bytes} [Wed Aug 31 19:05:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:24.813279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.851907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.861152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.927416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f7db3-d41b-42bb-a63b-099bc1b396a3 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.929795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.957508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2fccb-97e6-4031-bafd-b2eb15846a6d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:24.968356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bdf7d-901a-4ea0-9de4-5f08dfc22c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:24.969002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bdf7d-901a-4ea0-9de4-5f08dfc22c10 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.969280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bdf7d-901a-4ea0-9de4-5f08dfc22c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.969550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bdf7d-901a-4ea0-9de4-5f08dfc22c10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.970350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bdf7d-901a-4ea0-9de4-5f08dfc22c10 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:24.971738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bdf7d-901a-4ea0-9de4-5f08dfc22c10 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:24.996539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 510/1019] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:05:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:25.009652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150524d-e711-4281-9481-f0a290e5db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.011015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150524d-e711-4281-9481-f0a290e5db33 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.011508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150524d-e711-4281-9481-f0a290e5db33 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.011841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150524d-e711-4281-9481-f0a290e5db33 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.012537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150524d-e711-4281-9481-f0a290e5db33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:25.014024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150524d-e711-4281-9481-f0a290e5db33 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:25.032619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 510/1020] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:05:24 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:05:26.076180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff96adb-d881-4a43-9844-85176b938ab3 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.076778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff96adb-d881-4a43-9844-85176b938ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.076778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff96adb-d881-4a43-9844-85176b938ab3 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.078269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 511/1021] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:26.083299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2db80-5ca7-422f-a335-edcee60ce5e1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.083694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2db80-5ca7-422f-a335-edcee60ce5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.083857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2db80-5ca7-422f-a335-edcee60ce5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.102585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c1a2db80-5ca7-422f-a335-edcee60ce5e1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:26.104402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c1a2db80-5ca7-422f-a335-edcee60ce5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:26.176744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:26.195369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 511/1022] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:26.466203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff96adb-d881-4a43-9844-85176b938ab3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:26.574510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff96adb-d881-4a43-9844-85176b938ab3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:26.609738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f34ac3-a6b1-4d78-84ef-6678d2f37fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:26.610367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f34ac3-a6b1-4d78-84ef-6678d2f37fea None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.610648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f34ac3-a6b1-4d78-84ef-6678d2f37fea None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.610917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f34ac3-a6b1-4d78-84ef-6678d2f37fea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.612030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f34ac3-a6b1-4d78-84ef-6678d2f37fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:26.613495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f34ac3-a6b1-4d78-84ef-6678d2f37fea None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:26.615228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 512/1023] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:26.621586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2db80-5ca7-422f-a335-edcee60ce5e1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:26.724679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2db80-5ca7-422f-a335-edcee60ce5e1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:26.760326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3af8e-a83b-49bf-abf9-34443bf24060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:26.760968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3af8e-a83b-49bf-abf9-34443bf24060 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.761249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3af8e-a83b-49bf-abf9-34443bf24060 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.761523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3af8e-a83b-49bf-abf9-34443bf24060 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.762192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3af8e-a83b-49bf-abf9-34443bf24060 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:26.763632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3af8e-a83b-49bf-abf9-34443bf24060 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:26.792601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 512/1024] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:26 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:26.996308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1fea-75db-4563-8aa2-a54fa2cf6e6b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:26.996308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1fea-75db-4563-8aa2-a54fa2cf6e6b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:26.996308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1fea-75db-4563-8aa2-a54fa2cf6e6b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:26.996308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 513/1025] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:27.001650 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ae77c-9cf8-474a-864a-96f80403b241 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.001937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ae77c-9cf8-474a-864a-96f80403b241 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.002208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ae77c-9cf8-474a-864a-96f80403b241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.022035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-162ae77c-9cf8-474a-864a-96f80403b241 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:27.023224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-162ae77c-9cf8-474a-864a-96f80403b241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:27.065798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.086506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1528-53cc-4cd8-8f6e-7ea47478217c None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.087049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1528-53cc-4cd8-8f6e-7ea47478217c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.087049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1528-53cc-4cd8-8f6e-7ea47478217c None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.089378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 514/1026] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:27.094524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03243a9c-eb28-46be-969f-1d8c3ab2d3b6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.094806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03243a9c-eb28-46be-969f-1d8c3ab2d3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.095081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03243a9c-eb28-46be-969f-1d8c3ab2d3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.099717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 513/1027] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:27.113276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-03243a9c-eb28-46be-969f-1d8c3ab2d3b6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:27.114460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-03243a9c-eb28-46be-969f-1d8c3ab2d3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:27.143058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.193288 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 515/1028] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:27.366040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ae77c-9cf8-474a-864a-96f80403b241 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.464987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03243a9c-eb28-46be-969f-1d8c3ab2d3b6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.467137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ae77c-9cf8-474a-864a-96f80403b241 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.503681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.504317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.504692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.504970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] PATH_INFO: `/v3/services/image` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.505556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:27.506915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:27.516207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:27.517537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:27.518393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 514/1029] 10.222.0.95 () {56 vars in 1105 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:27.523168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.565923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.575959 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03243a9c-eb28-46be-969f-1d8c3ab2d3b6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.623003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77f303-4fc7-4357-b08a-6344a16848ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.623909 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f303-4fc7-4357-b08a-6344a16848ee None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.624430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f303-4fc7-4357-b08a-6344a16848ee None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.624912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f303-4fc7-4357-b08a-6344a16848ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.642047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77f303-4fc7-4357-b08a-6344a16848ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:27.650077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c9d26-11b1-448a-a8b8-b09a2dec2032 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-9d77f303-4fc7-4357-b08a-6344a16848ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:27.651815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:27.652772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:27.653617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:27.654285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 516/1030] 10.222.0.95 () {56 vars in 1120 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:27.662150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77f303-4fc7-4357-b08a-6344a16848ee None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.695782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.707539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.708432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.708926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.709383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.710428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] RBAC: Adding query filter params (name=image) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:27.710935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:27.713844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:27.729055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 515/1031] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:27.730538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.808732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77f303-4fc7-4357-b08a-6344a16848ee None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.848595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.849313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.849590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.849864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.850530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:27.850833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:27.851945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea929ed-04f4-43cb-80e2-49ef3eaf980a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.852220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:27.853548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.862333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 517/1032] 10.222.0.95 () {56 vars in 1115 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:27.869852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.894211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:27.894746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.895022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.895291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.895917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] RBAC: Adding query filter params (type=image) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:27.896218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:27.897621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:27.903438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 516/1033] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:27.909034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.957053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:27.978993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75220-1626-43e8-8161-4b1604b8809b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.016739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.017631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.017923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.018602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.027487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef0442-6627-4dc6-848c-e478881276cb None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.033194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.035155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:28.052721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:28.053747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:28.054675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:28.054675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:28.054675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:28.054675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 518/1034] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:28.060202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.066583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.067188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.067465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.067733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.068426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] RBAC: Adding query filter params (interface=public, service_id=7445ede59c224904b1febf694e5c03da, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:28.068756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.070105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.077266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 517/1035] 10.222.0.95 () {56 vars in 1251 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/endpoints?service_id=7445ede59c224904b1febf694e5c03da&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:28.082507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.115118 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.179622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559cac8-59c8-4612-bb7f-6e6d3db4b75c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.196376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17f24-4fd4-4b14-b9be-41bd4558b5d9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.209227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.215513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.216252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.216533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.216836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.217487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:28.217791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.219181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.231518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 519/1036] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:28.237031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.241050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae662b0-23d4-4ed6-812b-4a9fe7b4778d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.241658 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae662b0-23d4-4ed6-812b-4a9fe7b4778d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.241932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae662b0-23d4-4ed6-812b-4a9fe7b4778d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.242197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae662b0-23d4-4ed6-812b-4a9fe7b4778d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.242822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae662b0-23d4-4ed6-812b-4a9fe7b4778d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.248797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae662b0-23d4-4ed6-812b-4a9fe7b4778d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.264601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 518/1037] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/services => generated 1957 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:05:28.349423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0a1ae-391e-473f-82bc-f2158036993f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.387999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.388706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.388978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.389245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.397603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.398551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:28.412400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:28.413426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:28.414379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 520/1038] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:28.420176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae662b0-23d4-4ed6-812b-4a9fe7b4778d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.520846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.528562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae662b0-23d4-4ed6-812b-4a9fe7b4778d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.566231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43459c4-94e6-4bf3-b08f-ad59fa8b4342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.566852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43459c4-94e6-4bf3-b08f-ad59fa8b4342 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.567133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43459c4-94e6-4bf3-b08f-ad59fa8b4342 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.567407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43459c4-94e6-4bf3-b08f-ad59fa8b4342 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.568143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43459c4-94e6-4bf3-b08f-ad59fa8b4342 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:28.568451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43459c4-94e6-4bf3-b08f-ad59fa8b4342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.569976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43459c4-94e6-4bf3-b08f-ad59fa8b4342 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.592599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 519/1039] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:28.594525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.652926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.702747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3fa71-886b-46a6-98db-1a2d32feb498 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:28.739290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb943790-e757-4aa8-b214-4fbd7c6ab8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.739904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb943790-e757-4aa8-b214-4fbd7c6ab8ea None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.740187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb943790-e757-4aa8-b214-4fbd7c6ab8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.740459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb943790-e757-4aa8-b214-4fbd7c6ab8ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.810473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb943790-e757-4aa8-b214-4fbd7c6ab8ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.811857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb943790-e757-4aa8-b214-4fbd7c6ab8ea None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.863939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.882684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0759b8-bf66-4912-8ba7-4ef62e6ae85f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.883033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0759b8-bf66-4912-8ba7-4ef62e6ae85f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.883242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0759b8-bf66-4912-8ba7-4ef62e6ae85f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.884743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 520/1040] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:28.890465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4efb0-f732-47e9-b865-abca6b6fab68 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.890754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4efb0-f732-47e9-b865-abca6b6fab68 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.891028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4efb0-f732-47e9-b865-abca6b6fab68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.909658 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-adb4efb0-f732-47e9-b865-abca6b6fab68 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:28.910866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-adb4efb0-f732-47e9-b865-abca6b6fab68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:28.920874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 521/1041] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:28.926814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb943790-e757-4aa8-b214-4fbd7c6ab8ea None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:28.991688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 521/1042] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:29.039844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb943790-e757-4aa8-b214-4fbd7c6ab8ea None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.079514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.080428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.080919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.081387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.093084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.094870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:29.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:29.113233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:29.114136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:29.114136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:29.114136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:29.114136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 522/1043] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:29.120449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4efb0-f732-47e9-b865-abca6b6fab68 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.209970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.233068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4efb0-f732-47e9-b865-abca6b6fab68 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.268484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.269132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.269405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.269670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.270369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:29.270671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.272034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.277877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.283809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.286866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 522/1044] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:29.293469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.409250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a6997-1894-4d8a-80d1-6bb9890bdafc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.422491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2a6d9-c0d0-4315-9ec8-e6ed4a899168 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.446522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.448089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.448627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.449297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.456722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.458172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.459355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb5f38-95af-40fc-8572-32f7658c3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.459984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb5f38-95af-40fc-8572-32f7658c3b98 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.460266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb5f38-95af-40fc-8572-32f7658c3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.460541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb5f38-95af-40fc-8572-32f7658c3b98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:29.471078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:29.472178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:29.473322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:29.473322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:29.473322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:29.473322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:29.473322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:29.473322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:29.473322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:29.473322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:29.473857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb5f38-95af-40fc-8572-32f7658c3b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:29.475456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 523/1045] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-6efb5f38-95af-40fc-8572-32f7658c3b98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:29.476615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:29.477516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:29.478465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:29.481857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.482784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 523/1046] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:29.483522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb5f38-95af-40fc-8572-32f7658c3b98 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.485745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.547953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.602458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7cd24-67cf-4de7-afdd-33ec1fa9e11f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.607326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb5f38-95af-40fc-8572-32f7658c3b98 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.642411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.643026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.643304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.643570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.644257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:29.644562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.646021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.649042 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.649837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.650314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.650835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.651665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:29.652113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.653681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.659120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 524/1047] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:29.664743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 524/1048] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:29.665129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.670931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.735879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.780631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0510fa-da67-45da-861d-58c1af1d665b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.801578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824b4cf-31ed-4e65-84b9-101c51cf5ed2 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:29.821312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.822034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.823652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.823902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.825121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] RBAC: Adding query filter params (role.id=e7d6cbfad8444cc39b36987330c71769, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=aaab594854824542bd85426b0f16cbf6) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:29.825121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.826438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.831555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:29.832394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 525/1049] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/role_assignments?user.id=aaab594854824542bd85426b0f16cbf6&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=e7d6cbfad8444cc39b36987330c71769 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:29.845227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff2b0b-c96e-4da7-ba2e-5ccaac276302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:29.845898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff2b0b-c96e-4da7-ba2e-5ccaac276302 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:29.846252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff2b0b-c96e-4da7-ba2e-5ccaac276302 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:29.846525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff2b0b-c96e-4da7-ba2e-5ccaac276302 None admin] PATH_INFO: `/v3/users/c567d8c0b0ad4a569bfccd766145ab58` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:29.863504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff2b0b-c96e-4da7-ba2e-5ccaac276302 None admin] RBAC: Authorizing `identity:get_user(user_id=c567d8c0b0ad4a569bfccd766145ab58)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:29.865063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff2b0b-c96e-4da7-ba2e-5ccaac276302 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:29.867119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 525/1050] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/users/c567d8c0b0ad4a569bfccd766145ab58 => generated 285 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:29.873935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:29.990287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5c01d-22f2-405b-afad-79010549d21d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.018433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.031376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd78af-03bd-4b82-ae13-49b0abd78253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.032274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd78af-03bd-4b82-ae13-49b0abd78253 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.032877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd78af-03bd-4b82-ae13-49b0abd78253 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.033337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd78af-03bd-4b82-ae13-49b0abd78253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.120601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd78af-03bd-4b82-ae13-49b0abd78253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.122824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd78af-03bd-4b82-ae13-49b0abd78253 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.221884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c6601-31d6-404e-8fde-5f6f1475d13e None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.222237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c6601-31d6-404e-8fde-5f6f1475d13e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.222423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c6601-31d6-404e-8fde-5f6f1475d13e None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.223884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 526/1051] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:30.229102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38ff6-a920-4264-b3fa-8b4705000019 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.229389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38ff6-a920-4264-b3fa-8b4705000019 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.229660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38ff6-a920-4264-b3fa-8b4705000019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.236713 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 526/1052] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:29 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:30.242474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd78af-03bd-4b82-ae13-49b0abd78253 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.247722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-41e38ff6-a920-4264-b3fa-8b4705000019 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:30.249014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-41e38ff6-a920-4264-b3fa-8b4705000019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:30.276634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.325855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 527/1053] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:30.362527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd78af-03bd-4b82-ae13-49b0abd78253 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.406110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e07b4-a111-48e6-9ed8-9675a47a0d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.406740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e07b4-a111-48e6-9ed8-9675a47a0d0d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.407021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e07b4-a111-48e6-9ed8-9675a47a0d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.407293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e07b4-a111-48e6-9ed8-9675a47a0d0d None admin] PATH_INFO: `/v3/projects/3a2dfd9392a74edb87adc1cd427122bb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.408401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e07b4-a111-48e6-9ed8-9675a47a0d0d None admin] RBAC: Authorizing `identity:get_project(project_id=3a2dfd9392a74edb87adc1cd427122bb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.409877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e07b4-a111-48e6-9ed8-9675a47a0d0d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.416812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 527/1054] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/projects/3a2dfd9392a74edb87adc1cd427122bb => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:30.418911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e38ff6-a920-4264-b3fa-8b4705000019 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.524146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e38ff6-a920-4264-b3fa-8b4705000019 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.538522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.561470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.562126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.562412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.562684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.563407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=3a2dfd9392a74edb87adc1cd427122bb, user.id=c567d8c0b0ad4a569bfccd766145ab58) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:30.563714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.565183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.576825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 528/1055] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/role_assignments?user.id=c567d8c0b0ad4a569bfccd766145ab58&scope.project.id=3a2dfd9392a74edb87adc1cd427122bb&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:30.596679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e07b4-a111-48e6-9ed8-9675a47a0d0d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.702406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.711004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e07b4-a111-48e6-9ed8-9675a47a0d0d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.748595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.749230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.749508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.749778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] PATH_INFO: `/v3/services/image` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.750359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.751826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:30.761228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:30.762236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:30.763090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 528/1056] 10.222.0.95 () {56 vars in 1105 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:30.769906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.875412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550cc7f-04e1-4523-a8ab-16b1f428ce23 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.910118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.910744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.911021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.911291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.911943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] RBAC: Adding query filter params (name=image) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:30.912247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.913714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.920317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 529/1057] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:30.926123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:31.032811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0112fc7-7594-4916-b577-3db854f6d7ec None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:31.068333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab78c63-fae9-4c18-a06d-a661db5b1fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:31.068960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78c63-fae9-4c18-a06d-a661db5b1fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.069291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78c63-fae9-4c18-a06d-a661db5b1fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.069564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78c63-fae9-4c18-a06d-a661db5b1fc0 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.070180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab78c63-fae9-4c18-a06d-a661db5b1fc0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:31.070476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab78c63-fae9-4c18-a06d-a661db5b1fc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:31.071833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab78c63-fae9-4c18-a06d-a661db5b1fc0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:31.081718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 529/1058] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:31.082990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:31.133049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:31.198474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa637cb1-1f07-4336-ae85-64fb6cc2c474 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:31.233288 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ef959-94db-46ca-9f76-33b0c6f543d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:31.233900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ef959-94db-46ca-9f76-33b0c6f543d6 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.234179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ef959-94db-46ca-9f76-33b0c6f543d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.234451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ef959-94db-46ca-9f76-33b0c6f543d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.235116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ef959-94db-46ca-9f76-33b0c6f543d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:31.236501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ef959-94db-46ca-9f76-33b0c6f543d6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:31.256817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 530/1059] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:05:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:31.762919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429d3f-d0b1-41d3-bb2b-e7f5986a0d29 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.763459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429d3f-d0b1-41d3-bb2b-e7f5986a0d29 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.763459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429d3f-d0b1-41d3-bb2b-e7f5986a0d29 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.764980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 530/1060] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:31.770581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cb237-f68d-4fee-b338-38418032b18b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:31.771029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cb237-f68d-4fee-b338-38418032b18b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:31.771122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cb237-f68d-4fee-b338-38418032b18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:31.789499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-330cb237-f68d-4fee-b338-38418032b18b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:31.790685 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-330cb237-f68d-4fee-b338-38418032b18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:31.867542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 531/1061] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:32.133928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15429d3f-d0b1-41d3-bb2b-e7f5986a0d29 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:32.173234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:32.268331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15429d3f-d0b1-41d3-bb2b-e7f5986a0d29 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:32.303597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd164bb5-39bf-459f-af74-8e269f3ff473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:32.304226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd164bb5-39bf-459f-af74-8e269f3ff473 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:32.304502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd164bb5-39bf-459f-af74-8e269f3ff473 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:32.304800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd164bb5-39bf-459f-af74-8e269f3ff473 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:32.315763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd164bb5-39bf-459f-af74-8e269f3ff473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-bd164bb5-39bf-459f-af74-8e269f3ff473 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:32.320535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:32.321619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:32.323699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:32.327147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 531/1062] 10.222.0.95 () {56 vars in 1120 bytes} [Wed Aug 31 19:05:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:32.328708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cb237-f68d-4fee-b338-38418032b18b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:32.355273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:32.445738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cb237-f68d-4fee-b338-38418032b18b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:32.480331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6166c2f-6dc4-4195-b31f-c15b9b335763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:32.480964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6166c2f-6dc4-4195-b31f-c15b9b335763 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:32.481241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6166c2f-6dc4-4195-b31f-c15b9b335763 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:32.481507 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6166c2f-6dc4-4195-b31f-c15b9b335763 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:32.482167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6166c2f-6dc4-4195-b31f-c15b9b335763 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:32.482465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6166c2f-6dc4-4195-b31f-c15b9b335763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:32.483837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6166c2f-6dc4-4195-b31f-c15b9b335763 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:32.492185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 532/1063] 10.222.0.95 () {56 vars in 1115 bytes} [Wed Aug 31 19:05:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:32.498739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd164bb5-39bf-459f-af74-8e269f3ff473 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:32.527338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b896-275e-48b5-b0b7-d33b21f7a45b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:32.527656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b896-275e-48b5-b0b7-d33b21f7a45b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:32.527899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b896-275e-48b5-b0b7-d33b21f7a45b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:32.529397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 533/1064] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:32.534637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560c92f-f1be-4e08-af8c-fed2710e0354 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:32.534988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560c92f-f1be-4e08-af8c-fed2710e0354 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:32.535415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560c92f-f1be-4e08-af8c-fed2710e0354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:32.554705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a560c92f-f1be-4e08-af8c-fed2710e0354 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:32.555906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a560c92f-f1be-4e08-af8c-fed2710e0354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:32.605833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:32.640411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd164bb5-39bf-459f-af74-8e269f3ff473 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:32.644585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 534/1065] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:32.679930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:32.680773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:32.681060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:32.681335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:32.694052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:32.695627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:32.712006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:32.718727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:32.719610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:32.719610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:32.719610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:32.719610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 532/1066] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:32.721518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560c92f-f1be-4e08-af8c-fed2710e0354 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:32.740069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:32.830665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560c92f-f1be-4e08-af8c-fed2710e0354 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:32.865594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:32.866201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:32.866479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:32.866748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:32.867396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:32.867695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:32.869757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:32.887030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 535/1067] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:32.888550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:32.917477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.007450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d0f82-49da-4943-9fc0-e2fbb4f41b97 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.011728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.039323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dad57-57c6-4150-9b26-42d10b76973b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.052943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.053757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.054110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.054445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.057462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.066766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.068520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.079723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3699683-2999-4eb7-a1f4-a50abe69c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.080557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3699683-2999-4eb7-a1f4-a50abe69c401 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.080860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3699683-2999-4eb7-a1f4-a50abe69c401 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.081136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3699683-2999-4eb7-a1f4-a50abe69c401 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:33.084061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:33.085304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:33.086795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:33.091610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 533/1068] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:05:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:33.092858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.098479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3699683-2999-4eb7-a1f4-a50abe69c401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-e3699683-2999-4eb7-a1f4-a50abe69c401 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:33.103682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:33.105535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:33.106642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 536/1069] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:33.111722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3699683-2999-4eb7-a1f4-a50abe69c401 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.226508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed970c-114d-43f6-a65b-3118c51885c7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.248831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3699683-2999-4eb7-a1f4-a50abe69c401 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.265244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af94f73-bdf5-476a-af8b-0bdd8e34815c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.265784 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af94f73-bdf5-476a-af8b-0bdd8e34815c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.266070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af94f73-bdf5-476a-af8b-0bdd8e34815c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.266340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af94f73-bdf5-476a-af8b-0bdd8e34815c None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.267065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af94f73-bdf5-476a-af8b-0bdd8e34815c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:33.267447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af94f73-bdf5-476a-af8b-0bdd8e34815c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.268839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af94f73-bdf5-476a-af8b-0bdd8e34815c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.284226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793780-b648-43f6-a5f4-bfae0dcb31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.284856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793780-b648-43f6-a5f4-bfae0dcb31cc None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.285137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793780-b648-43f6-a5f4-bfae0dcb31cc None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.285403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793780-b648-43f6-a5f4-bfae0dcb31cc None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.286110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793780-b648-43f6-a5f4-bfae0dcb31cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:33.288688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793780-b648-43f6-a5f4-bfae0dcb31cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.288785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 534/1070] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:33.288785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161024d-c974-4191-b171-9abb1e2c47aa None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.288939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793780-b648-43f6-a5f4-bfae0dcb31cc None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.289026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161024d-c974-4191-b171-9abb1e2c47aa None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.289026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161024d-c974-4191-b171-9abb1e2c47aa None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.289745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 535/1071] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:33.292796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161024d-c974-4191-b171-9abb1e2c47aa None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.294121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.300847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 537/1072] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:33.301999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a256e-a241-451d-86a1-9b02cfec434b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.302284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a256e-a241-451d-86a1-9b02cfec434b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.302557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a256e-a241-451d-86a1-9b02cfec434b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.321749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fe7a256e-a241-451d-86a1-9b02cfec434b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:33.323642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fe7a256e-a241-451d-86a1-9b02cfec434b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:33.349329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.392050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 538/1073] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:33.392950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a256e-a241-451d-86a1-9b02cfec434b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.398815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161024d-c974-4191-b171-9abb1e2c47aa None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.442295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.443207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.443490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.443760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.457769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.459508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:33.479082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:33.480239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:33.486440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:33.486440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:33.486440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:33.486440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 536/1074] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:33.486884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.498465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a256e-a241-451d-86a1-9b02cfec434b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.531454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.543491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b075b14-2809-42cd-aeb4-7c26bb3f66fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.544334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b075b14-2809-42cd-aeb4-7c26bb3f66fe None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.544646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b075b14-2809-42cd-aeb4-7c26bb3f66fe None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.544922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b075b14-2809-42cd-aeb4-7c26bb3f66fe None admin] PATH_INFO: `/v3/users/c567d8c0b0ad4a569bfccd766145ab58` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.546875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b075b14-2809-42cd-aeb4-7c26bb3f66fe None admin] RBAC: Authorizing `identity:get_user(user_id=c567d8c0b0ad4a569bfccd766145ab58)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.549130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b075b14-2809-42cd-aeb4-7c26bb3f66fe None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.555589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 539/1075] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/users/c567d8c0b0ad4a569bfccd766145ab58 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:33.557444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b075b14-2809-42cd-aeb4-7c26bb3f66fe None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.614991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4674863-1838-4002-a78c-98a935b702f1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.618220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.654652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.655544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.655827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.656100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.656797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:33.657103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.658906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.670937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b075b14-2809-42cd-aeb4-7c26bb3f66fe None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.671733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 537/1076] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:33.672549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.706823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139f128-d78b-4ef0-beb3-098bdbe6b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.707452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139f128-d78b-4ef0-beb3-098bdbe6b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.707736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139f128-d78b-4ef0-beb3-098bdbe6b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.708018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139f128-d78b-4ef0-beb3-098bdbe6b6b7 None admin] PATH_INFO: `/v3/projects/3a2dfd9392a74edb87adc1cd427122bb` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.709172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139f128-d78b-4ef0-beb3-098bdbe6b6b7 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2dfd9392a74edb87adc1cd427122bb)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.710620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139f128-d78b-4ef0-beb3-098bdbe6b6b7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.712254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 540/1077] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/projects/3a2dfd9392a74edb87adc1cd427122bb => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:33.713143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139f128-d78b-4ef0-beb3-098bdbe6b6b7 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.776361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.777851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714159-8a2a-4932-a2ae-e7b03cf28ec1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.814350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124ffef-b49f-4454-8a2b-9cd057a3b036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.814874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124ffef-b49f-4454-8a2b-9cd057a3b036 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.815155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124ffef-b49f-4454-8a2b-9cd057a3b036 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.815425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124ffef-b49f-4454-8a2b-9cd057a3b036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.815842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139f128-d78b-4ef0-beb3-098bdbe6b6b7 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.845938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.849332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.849955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.850758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.850758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.857746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.859210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:33.870666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:33.871749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 541/1078] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:33.872708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.887967 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124ffef-b49f-4454-8a2b-9cd057a3b036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.889790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124ffef-b49f-4454-8a2b-9cd057a3b036 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.977235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc36ba2-c7ee-4f1c-afc4-3ef3ad50b9e5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.989612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 538/1079] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:33.990447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124ffef-b49f-4454-8a2b-9cd057a3b036 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:34.001215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.013319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878dffa-744a-4d3f-bc71-a720bd932d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:34.013979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878dffa-744a-4d3f-bc71-a720bd932d10 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.014259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878dffa-744a-4d3f-bc71-a720bd932d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.014531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878dffa-744a-4d3f-bc71-a720bd932d10 None admin] PATH_INFO: `/v3/projects/3a2dfd9392a74edb87adc1cd427122bb/users/c567d8c0b0ad4a569bfccd766145ab58/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.016464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878dffa-744a-4d3f-bc71-a720bd932d10 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a2dfd9392a74edb87adc1cd427122bb, user_id=c567d8c0b0ad4a569bfccd766145ab58, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:34.017968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878dffa-744a-4d3f-bc71-a720bd932d10 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:34.032077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 542/1080] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:33 2022] PUT /identity/v3/projects/3a2dfd9392a74edb87adc1cd427122bb/users/c567d8c0b0ad4a569bfccd766145ab58/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:34.032724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878dffa-744a-4d3f-bc71-a720bd932d10 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:34.062388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.110076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124ffef-b49f-4454-8a2b-9cd057a3b036 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:34.157243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:34.157860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.158137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.158405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.159109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:34.159411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:34.160886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:34.165067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878dffa-744a-4d3f-bc71-a720bd932d10 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:34.177581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 539/1081] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:34.183625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:34.208110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:34.208893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.209141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.209346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.222491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:34.223955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:34.239067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:34.240096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:34.241032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:34.241032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:34.241032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:34.241032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 543/1082] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:34.248680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:34.288690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.303603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8c79b-a1f9-40f4-a78b-c4e22a6427d0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:34.330013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.341451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccfea4-f1a0-4ffa-a576-a598db60ffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:34.342151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccfea4-f1a0-4ffa-a576-a598db60ffa7 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.342433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccfea4-f1a0-4ffa-a576-a598db60ffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.342711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccfea4-f1a0-4ffa-a576-a598db60ffa7 None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/aaab594854824542bd85426b0f16cbf6/roles/e7d6cbfad8444cc39b36987330c71769` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.344961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccfea4-f1a0-4ffa-a576-a598db60ffa7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=aaab594854824542bd85426b0f16cbf6, role_id=e7d6cbfad8444cc39b36987330c71769)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:34.346414 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccfea4-f1a0-4ffa-a576-a598db60ffa7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:34.372595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 540/1083] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:34 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/aaab594854824542bd85426b0f16cbf6/roles/e7d6cbfad8444cc39b36987330c71769 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:05:34.372863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0d74c-6b9e-4f49-9541-e21fa6e8f88f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:34.412686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2eb50-e37b-459a-9820-51f4ebd30ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:34.413412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2eb50-e37b-459a-9820-51f4ebd30ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.413773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2eb50-e37b-459a-9820-51f4ebd30ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.414117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2eb50-e37b-459a-9820-51f4ebd30ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.414868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2eb50-e37b-459a-9820-51f4ebd30ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:34.415246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2eb50-e37b-459a-9820-51f4ebd30ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:34.417553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2eb50-e37b-459a-9820-51f4ebd30ed6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:34.440643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 544/1084] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:35.954724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe84c-f39e-467b-a5e8-76cf756289d8 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.955284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe84c-f39e-467b-a5e8-76cf756289d8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.955384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe84c-f39e-467b-a5e8-76cf756289d8 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.956761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 541/1085] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:35.961946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee84de-43cb-43c4-9dac-a596e3339a09 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.962402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee84de-43cb-43c4-9dac-a596e3339a09 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.962507 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee84de-43cb-43c4-9dac-a596e3339a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.981478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-29ee84de-43cb-43c4-9dac-a596e3339a09 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:35.982666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-29ee84de-43cb-43c4-9dac-a596e3339a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:36.047626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 545/1086] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:36.287050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576accab-18f8-4c64-9df9-c523bb2a61a7 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.287330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576accab-18f8-4c64-9df9-c523bb2a61a7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.287597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576accab-18f8-4c64-9df9-c523bb2a61a7 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.289036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 542/1087] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:36.294849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ef2b9-bf48-423d-868a-32fdd910771f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.295140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ef2b9-bf48-423d-868a-32fdd910771f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.295412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ef2b9-bf48-423d-868a-32fdd910771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.313587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4e0ef2b9-bf48-423d-868a-32fdd910771f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:36.314861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4e0ef2b9-bf48-423d-868a-32fdd910771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:36.315324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576accab-18f8-4c64-9df9-c523bb2a61a7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.336045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.389452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 546/1088] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:36.417983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576accab-18f8-4c64-9df9-c523bb2a61a7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.451701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16f7ca-0266-4bef-a2b3-0a57dbc660a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.452321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16f7ca-0266-4bef-a2b3-0a57dbc660a2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.452624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16f7ca-0266-4bef-a2b3-0a57dbc660a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.452900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16f7ca-0266-4bef-a2b3-0a57dbc660a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.463338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea16f7ca-0266-4bef-a2b3-0a57dbc660a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ea16f7ca-0266-4bef-a2b3-0a57dbc660a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:36.468106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:36.469313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:36.470230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 543/1089] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:36.475261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ef2b9-bf48-423d-868a-32fdd910771f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.574370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ef2b9-bf48-423d-868a-32fdd910771f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.610664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.611385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.611580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.611927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.612774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:36.613133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.614722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.625092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 547/1090] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:36.630868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16f7ca-0266-4bef-a2b3-0a57dbc660a2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.659605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.675809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.679447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.733529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16f7ca-0266-4bef-a2b3-0a57dbc660a2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.761075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c8a2c-2898-4deb-8073-7e162bceed23 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.767372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806508c1-bad6-477d-acd8-9d8768267dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.767997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806508c1-bad6-477d-acd8-9d8768267dbd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.768275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806508c1-bad6-477d-acd8-9d8768267dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.768548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806508c1-bad6-477d-acd8-9d8768267dbd None admin] PATH_INFO: `/v3/users/c567d8c0b0ad4a569bfccd766145ab58` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.769691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806508c1-bad6-477d-acd8-9d8768267dbd None admin] RBAC: Authorizing `identity:get_user(user_id=c567d8c0b0ad4a569bfccd766145ab58)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.771116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806508c1-bad6-477d-acd8-9d8768267dbd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.772745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 544/1091] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/users/c567d8c0b0ad4a569bfccd766145ab58 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:36.779092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806508c1-bad6-477d-acd8-9d8768267dbd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.794978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718ba38-d742-4dc8-90cd-65eae3cc3317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.795599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718ba38-d742-4dc8-90cd-65eae3cc3317 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.795873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718ba38-d742-4dc8-90cd-65eae3cc3317 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.796143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718ba38-d742-4dc8-90cd-65eae3cc3317 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.808044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718ba38-d742-4dc8-90cd-65eae3cc3317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-e718ba38-d742-4dc8-90cd-65eae3cc3317 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:36.813275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:36.814274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:36.815292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 548/1092] 10.222.0.95 () {56 vars in 1120 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:36.820173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718ba38-d742-4dc8-90cd-65eae3cc3317 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.881923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806508c1-bad6-477d-acd8-9d8768267dbd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.894947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.895603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.916897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cabcf-a3f1-406c-9e86-faea684ffb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.917525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cabcf-a3f1-406c-9e86-faea684ffb00 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.917808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cabcf-a3f1-406c-9e86-faea684ffb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.917935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718ba38-d742-4dc8-90cd-65eae3cc3317 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.918099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cabcf-a3f1-406c-9e86-faea684ffb00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.951753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.952340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.952639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.952909 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.953560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:36.953855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.955222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.963783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 549/1093] 10.222.0.95 () {56 vars in 1115 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:36.969241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.990633 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458cabcf-a3f1-406c-9e86-faea684ffb00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.992093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458cabcf-a3f1-406c-9e86-faea684ffb00 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:37.070341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cdb9d-3e62-48fc-94d5-b8809a9142f9 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:37.092471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 545/1094] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 19:05:37.098428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cabcf-a3f1-406c-9e86-faea684ffb00 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:37.103659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:37.104293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.104605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.104878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.112286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:37.114477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:37.116750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:37.118456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:37.141261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:37.143454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:37.144788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:37.144788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:37.144788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:37.144788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 550/1095] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:05:37.148623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:37.214890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cabcf-a3f1-406c-9e86-faea684ffb00 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:37.248991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4e52b-4054-4bf4-8c04-e3d3d3ef4ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:37.249518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4407e-e5a6-4116-ab61-67b738c4da38 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:37.249647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4e52b-4054-4bf4-8c04-e3d3d3ef4ddb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.249889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4e52b-4054-4bf4-8c04-e3d3d3ef4ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.250152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4e52b-4054-4bf4-8c04-e3d3d3ef4ddb None admin] PATH_INFO: `/v3/projects/3a2dfd9392a74edb87adc1cd427122bb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.251243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4e52b-4054-4bf4-8c04-e3d3d3ef4ddb None admin] RBAC: Authorizing `identity:get_project(project_id=3a2dfd9392a74edb87adc1cd427122bb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:37.252700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4e52b-4054-4bf4-8c04-e3d3d3ef4ddb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:37.254344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 546/1096] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:37 2022] GET /identity/v3/projects/3a2dfd9392a74edb87adc1cd427122bb => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:37.261267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4e52b-4054-4bf4-8c04-e3d3d3ef4ddb None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:37.286138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:37.286742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.287018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.287283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.287919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:37.288214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:37.289610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:37.302164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 551/1097] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:37.307868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:37.341503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:37.360028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:37.362013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4e52b-4054-4bf4-8c04-e3d3d3ef4ddb None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:37.397874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:37.398421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.398703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.398971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.399718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=3a2dfd9392a74edb87adc1cd427122bb, user.id=c567d8c0b0ad4a569bfccd766145ab58) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:37.400081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:37.402283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:37.408099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 547/1098] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:37 2022] GET /identity/v3/role_assignments?user.id=c567d8c0b0ad4a569bfccd766145ab58&scope.project.id=3a2dfd9392a74edb87adc1cd427122bb&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:37.412151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759ce3b-2f7b-4df7-b706-8c933aaff8ed None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:37.450661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:37.451308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.451588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.451859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.460071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:37.461794 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:05:37.476234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:37.477485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:37.478645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 552/1099] 10.222.0.95 () {56 vars in 1093 bytes} [Wed Aug 31 19:05:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:37.483935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:37.589465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2e13e-1578-44a8-beda-337d80ac3da2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:37.624515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:37.625244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.625400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.625670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.626419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:37.626716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:37.628142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:37.646802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 548/1100] 10.222.0.95 () {56 vars in 1133 bytes} [Wed Aug 31 19:05:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:37.653650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:37.765425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724849a-dcda-43f8-8a67-5e8248375360 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:37.779297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:37.802168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe3e29-b9c9-47db-92a9-98d6309b6644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:37.802839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe3e29-b9c9-47db-92a9-98d6309b6644 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:37.803124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe3e29-b9c9-47db-92a9-98d6309b6644 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:37.803394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe3e29-b9c9-47db-92a9-98d6309b6644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:37.878572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe3e29-b9c9-47db-92a9-98d6309b6644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:37.880040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe3e29-b9c9-47db-92a9-98d6309b6644 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:37.977975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 553/1101] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:37.983999 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:37.986973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.097936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c1d84-b400-4ba3-9b4b-61e8308f3d6c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.147533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.148492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.148638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.148838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.160447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:38.161958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:05:38.177489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:38.178636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:05:38.179527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:05:38.179527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:05:38.179527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:38.179527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 549/1102] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:38.185095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe3e29-b9c9-47db-92a9-98d6309b6644 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.201921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.294116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe3e29-b9c9-47db-92a9-98d6309b6644 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.334067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.334678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.334960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.335233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.335886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:38.336189 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:38.338302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:38.350308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 554/1103] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:38.356382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.424157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.477301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88d36c-92f2-4ff2-96e5-4af206fae202 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.517463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.518089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.518364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.518667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.526023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:38.527610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:05:38.540503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:38.541638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:05:38.542796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 550/1104] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:38.549452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.630975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.661881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707e809-57cf-4bb8-8b2b-28f0676c2256 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.701736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f2d7c-31a4-4c33-baeb-9b015ac1f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.702362 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f2d7c-31a4-4c33-baeb-9b015ac1f601 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.702674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f2d7c-31a4-4c33-baeb-9b015ac1f601 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.702966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f2d7c-31a4-4c33-baeb-9b015ac1f601 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.703697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f2d7c-31a4-4c33-baeb-9b015ac1f601 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:38.704010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f2d7c-31a4-4c33-baeb-9b015ac1f601 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:38.705544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f2d7c-31a4-4c33-baeb-9b015ac1f601 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:38.728709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 555/1105] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:38.730168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:38.845419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c39f31-48fd-452d-82d2-36cd9a1cb15f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:38.861901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:38.884792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fed76f-39df-4954-8d76-4572809722dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:38.885423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fed76f-39df-4954-8d76-4572809722dd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.885700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fed76f-39df-4954-8d76-4572809722dd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.886137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fed76f-39df-4954-8d76-4572809722dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.886695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fed76f-39df-4954-8d76-4572809722dd None admin] RBAC: Adding query filter params (role.id=e7d6cbfad8444cc39b36987330c71769, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=aaab594854824542bd85426b0f16cbf6) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:38.886997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fed76f-39df-4954-8d76-4572809722dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:38.888424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fed76f-39df-4954-8d76-4572809722dd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:38.894492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 551/1106] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:38 2022] GET /identity/v3/role_assignments?user.id=aaab594854824542bd85426b0f16cbf6&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=e7d6cbfad8444cc39b36987330c71769 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:39.439136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5bb37-7b68-4947-8d03-456a956ea23c None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.439648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5bb37-7b68-4947-8d03-456a956ea23c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.439648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5bb37-7b68-4947-8d03-456a956ea23c None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.441029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 556/1107] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:39.446678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723455b-9dad-40cd-af67-1af8e8fd49b2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.447289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723455b-9dad-40cd-af67-1af8e8fd49b2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.447289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723455b-9dad-40cd-af67-1af8e8fd49b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.466929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0723455b-9dad-40cd-af67-1af8e8fd49b2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:39.468160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0723455b-9dad-40cd-af67-1af8e8fd49b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:39.551050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 552/1108] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:39.827438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5bb37-7b68-4947-8d03-456a956ea23c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.927554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5bb37-7b68-4947-8d03-456a956ea23c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.957122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.962414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc9e60-5368-4753-b990-d2aaeab27662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.963591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc9e60-5368-4753-b990-d2aaeab27662 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.963885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc9e60-5368-4753-b990-d2aaeab27662 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.964152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc9e60-5368-4753-b990-d2aaeab27662 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.965276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc9e60-5368-4753-b990-d2aaeab27662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.966710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc9e60-5368-4753-b990-d2aaeab27662 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.968259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 557/1109] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:39.974059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723455b-9dad-40cd-af67-1af8e8fd49b2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:40.074881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723455b-9dad-40cd-af67-1af8e8fd49b2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:40.084281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:40.111062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12505aeb-8488-4df1-a362-baaee668fc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:40.111667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12505aeb-8488-4df1-a362-baaee668fc0d None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:40.111953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12505aeb-8488-4df1-a362-baaee668fc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:40.112230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12505aeb-8488-4df1-a362-baaee668fc0d None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:40.112935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12505aeb-8488-4df1-a362-baaee668fc0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:40.114393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12505aeb-8488-4df1-a362-baaee668fc0d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:40.144899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 553/1110] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:39 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:42.118927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9fefd-acf6-4859-8cef-7ed6d8e9e3e2 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.120074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9fefd-acf6-4859-8cef-7ed6d8e9e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.120074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9fefd-acf6-4859-8cef-7ed6d8e9e3e2 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.121484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 558/1111] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:42.127737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03f3d0-5ddd-4e2a-b28d-6c7de02d85dc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.128208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03f3d0-5ddd-4e2a-b28d-6c7de02d85dc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.128711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03f3d0-5ddd-4e2a-b28d-6c7de02d85dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.150705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6b03f3d0-5ddd-4e2a-b28d-6c7de02d85dc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:42.151917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6b03f3d0-5ddd-4e2a-b28d-6c7de02d85dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:42.220217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 554/1112] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:42.496966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9fefd-acf6-4859-8cef-7ed6d8e9e3e2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.595398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9fefd-acf6-4859-8cef-7ed6d8e9e3e2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.632285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e23cd-4631-4023-9816-32d7da97019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.632928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e23cd-4631-4023-9816-32d7da97019b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.633206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e23cd-4631-4023-9816-32d7da97019b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.633474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e23cd-4631-4023-9816-32d7da97019b None admin] PATH_INFO: `/v3/roles/79dbf286cf884619af62f33ea75c52bf` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.635429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e23cd-4631-4023-9816-32d7da97019b None admin] RBAC: Authorizing `identity:get_role(role_id=79dbf286cf884619af62f33ea75c52bf)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.636822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e23cd-4631-4023-9816-32d7da97019b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.638134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 559/1113] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/roles/79dbf286cf884619af62f33ea75c52bf => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:42.644047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03f3d0-5ddd-4e2a-b28d-6c7de02d85dc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.737226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.745351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03f3d0-5ddd-4e2a-b28d-6c7de02d85dc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.780447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3833a-cd79-42e3-aeca-a0497d799165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.781018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3833a-cd79-42e3-aeca-a0497d799165 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.781298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3833a-cd79-42e3-aeca-a0497d799165 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.781570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3833a-cd79-42e3-aeca-a0497d799165 None admin] PATH_INFO: `/v3/users/dbe9b2a9fbdb4cde9590d9b400ac065a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.799349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3833a-cd79-42e3-aeca-a0497d799165 None admin] RBAC: Authorizing `identity:get_user(user_id=dbe9b2a9fbdb4cde9590d9b400ac065a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.800835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3833a-cd79-42e3-aeca-a0497d799165 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.802528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 555/1114] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/users/dbe9b2a9fbdb4cde9590d9b400ac065a => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:42.808867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e23cd-4631-4023-9816-32d7da97019b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.879898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.916825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e23cd-4631-4023-9816-32d7da97019b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.951182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07f892-e901-43f5-8d14-8c9fbe96d9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.951788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07f892-e901-43f5-8d14-8c9fbe96d9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.952070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07f892-e901-43f5-8d14-8c9fbe96d9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.952342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07f892-e901-43f5-8d14-8c9fbe96d9c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.019331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07f892-e901-43f5-8d14-8c9fbe96d9c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.020862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07f892-e901-43f5-8d14-8c9fbe96d9c5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.084600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:43.117643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 560/1115] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:43.124355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3833a-cd79-42e3-aeca-a0497d799165 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:43.227530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3833a-cd79-42e3-aeca-a0497d799165 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:43.261863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18a2c-f6db-4385-a7b7-10d9b0377a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:43.262496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18a2c-f6db-4385-a7b7-10d9b0377a80 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.262781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18a2c-f6db-4385-a7b7-10d9b0377a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.263058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18a2c-f6db-4385-a7b7-10d9b0377a80 None admin] PATH_INFO: `/v3/projects/3a2dfd9392a74edb87adc1cd427122bb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.264174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c18a2c-f6db-4385-a7b7-10d9b0377a80 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2dfd9392a74edb87adc1cd427122bb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.265646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c18a2c-f6db-4385-a7b7-10d9b0377a80 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.267284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 556/1116] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:43 2022] GET /identity/v3/projects/3a2dfd9392a74edb87adc1cd427122bb => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:43.273527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07f892-e901-43f5-8d14-8c9fbe96d9c5 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:43.376523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07f892-e901-43f5-8d14-8c9fbe96d9c5 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:43.412066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee264a-9171-48ad-91f0-f46ddfa4f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:43.412701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee264a-9171-48ad-91f0-f46ddfa4f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:43.412983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee264a-9171-48ad-91f0-f46ddfa4f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:43.413266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee264a-9171-48ad-91f0-f46ddfa4f2f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:43.413965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee264a-9171-48ad-91f0-f46ddfa4f2f2 None admin] RBAC: Adding query filter params (role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=3a2dfd9392a74edb87adc1cd427122bb, user.id=dbe9b2a9fbdb4cde9590d9b400ac065a) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:43.414271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee264a-9171-48ad-91f0-f46ddfa4f2f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:43.416242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee264a-9171-48ad-91f0-f46ddfa4f2f2 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:43.421518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 561/1117] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:43 2022] GET /identity/v3/role_assignments?user.id=dbe9b2a9fbdb4cde9590d9b400ac065a&scope.project.id=3a2dfd9392a74edb87adc1cd427122bb&role.id=79dbf286cf884619af62f33ea75c52bf => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:45.306641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe875b96-da31-4483-b01e-9e05e1fd3ac6 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.307200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe875b96-da31-4483-b01e-9e05e1fd3ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.307200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe875b96-da31-4483-b01e-9e05e1fd3ac6 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.308546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 557/1118] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:45.313581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac056-8dbe-45d4-b271-a437cd1f9a5b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.314027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac056-8dbe-45d4-b271-a437cd1f9a5b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.314131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ac056-8dbe-45d4-b271-a437cd1f9a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.332466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-132ac056-8dbe-45d4-b271-a437cd1f9a5b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:45.333694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-132ac056-8dbe-45d4-b271-a437cd1f9a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:45.360537 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:45.400990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 562/1119] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:45.670169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe875b96-da31-4483-b01e-9e05e1fd3ac6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:45.691519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:45.771754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe875b96-da31-4483-b01e-9e05e1fd3ac6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.804796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae560e9-5561-43e3-bb0c-e5d2820315f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.805407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae560e9-5561-43e3-bb0c-e5d2820315f8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.805682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae560e9-5561-43e3-bb0c-e5d2820315f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.805965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae560e9-5561-43e3-bb0c-e5d2820315f8 None admin] PATH_INFO: `/v3/roles/79dbf286cf884619af62f33ea75c52bf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.807355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae560e9-5561-43e3-bb0c-e5d2820315f8 None admin] RBAC: Authorizing `identity:get_role(role_id=79dbf286cf884619af62f33ea75c52bf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:45.808824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae560e9-5561-43e3-bb0c-e5d2820315f8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:45.810082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 558/1120] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/roles/79dbf286cf884619af62f33ea75c52bf => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:45.815963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ac056-8dbe-45d4-b271-a437cd1f9a5b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:45.919164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ac056-8dbe-45d4-b271-a437cd1f9a5b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:45.953761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84187758-7629-4ef9-9a3f-9a1226f58469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:45.954309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187758-7629-4ef9-9a3f-9a1226f58469 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:45.954588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187758-7629-4ef9-9a3f-9a1226f58469 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:45.954859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187758-7629-4ef9-9a3f-9a1226f58469 None admin] PATH_INFO: `/v3/users/dbe9b2a9fbdb4cde9590d9b400ac065a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:45.955977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84187758-7629-4ef9-9a3f-9a1226f58469 None admin] RBAC: Authorizing `identity:get_user(user_id=dbe9b2a9fbdb4cde9590d9b400ac065a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:45.957448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84187758-7629-4ef9-9a3f-9a1226f58469 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:45.959023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 563/1121] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/users/dbe9b2a9fbdb4cde9590d9b400ac065a => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:45.965638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae560e9-5561-43e3-bb0c-e5d2820315f8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.054264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:46.067386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae560e9-5561-43e3-bb0c-e5d2820315f8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.101423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17055204-5672-46d5-be2d-ede7755f2db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.102220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055204-5672-46d5-be2d-ede7755f2db3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.102501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055204-5672-46d5-be2d-ede7755f2db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.102777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055204-5672-46d5-be2d-ede7755f2db3 None admin] PATH_INFO: `/v3/projects/3a2dfd9392a74edb87adc1cd427122bb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.103909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17055204-5672-46d5-be2d-ede7755f2db3 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2dfd9392a74edb87adc1cd427122bb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.105529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17055204-5672-46d5-be2d-ede7755f2db3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.107198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 559/1122] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:45 2022] GET /identity/v3/projects/3a2dfd9392a74edb87adc1cd427122bb => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:05:46.113464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84187758-7629-4ef9-9a3f-9a1226f58469 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:46.144608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:46.214587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84187758-7629-4ef9-9a3f-9a1226f58469 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:46.248525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e96911-b6e7-49b4-83aa-a3ac4002bc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:46.249204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e96911-b6e7-49b4-83aa-a3ac4002bc8d None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:46.249481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e96911-b6e7-49b4-83aa-a3ac4002bc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:46.249751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e96911-b6e7-49b4-83aa-a3ac4002bc8d None admin] PATH_INFO: `/v3/projects/3a2dfd9392a74edb87adc1cd427122bb/users/dbe9b2a9fbdb4cde9590d9b400ac065a/roles/79dbf286cf884619af62f33ea75c52bf` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:46.251916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e96911-b6e7-49b4-83aa-a3ac4002bc8d None admin] RBAC: Authorizing `identity:create_grant(project_id=3a2dfd9392a74edb87adc1cd427122bb, user_id=dbe9b2a9fbdb4cde9590d9b400ac065a, role_id=79dbf286cf884619af62f33ea75c52bf)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:46.253404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e96911-b6e7-49b4-83aa-a3ac4002bc8d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:46.266569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 564/1123] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:05:46 2022] PUT /identity/v3/projects/3a2dfd9392a74edb87adc1cd427122bb/users/dbe9b2a9fbdb4cde9590d9b400ac065a/roles/79dbf286cf884619af62f33ea75c52bf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:05:48.160876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a8a84-2635-4797-b48b-ed30837ddbe7 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.161383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a8a84-2635-4797-b48b-ed30837ddbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.161475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a8a84-2635-4797-b48b-ed30837ddbe7 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.162792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 560/1124] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:48.168532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e73d9-8635-4fb9-8448-721671e4e32b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.168984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e73d9-8635-4fb9-8448-721671e4e32b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.169101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e73d9-8635-4fb9-8448-721671e4e32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.189086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d83e73d9-8635-4fb9-8448-721671e4e32b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:48.190272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d83e73d9-8635-4fb9-8448-721671e4e32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:48.256344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 565/1125] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:48.525671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a8a84-2635-4797-b48b-ed30837ddbe7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.623696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a8a84-2635-4797-b48b-ed30837ddbe7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.657461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da796213-10ec-44ed-9d93-5adb9c892c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.658676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da796213-10ec-44ed-9d93-5adb9c892c3b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.658676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da796213-10ec-44ed-9d93-5adb9c892c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.658906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da796213-10ec-44ed-9d93-5adb9c892c3b None admin] PATH_INFO: `/v3/roles/79dbf286cf884619af62f33ea75c52bf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.660423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da796213-10ec-44ed-9d93-5adb9c892c3b None admin] RBAC: Authorizing `identity:get_role(role_id=79dbf286cf884619af62f33ea75c52bf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.662076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da796213-10ec-44ed-9d93-5adb9c892c3b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.663410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 561/1126] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/roles/79dbf286cf884619af62f33ea75c52bf => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:48.669538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e73d9-8635-4fb9-8448-721671e4e32b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.690302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.768726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e73d9-8635-4fb9-8448-721671e4e32b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.802080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc787a4-82a4-4a1c-a662-e69ca6f3b71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.802628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc787a4-82a4-4a1c-a662-e69ca6f3b71f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.802902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc787a4-82a4-4a1c-a662-e69ca6f3b71f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.803168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc787a4-82a4-4a1c-a662-e69ca6f3b71f None admin] PATH_INFO: `/v3/users/dbe9b2a9fbdb4cde9590d9b400ac065a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:48.804301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc787a4-82a4-4a1c-a662-e69ca6f3b71f None admin] RBAC: Authorizing `identity:get_user(user_id=dbe9b2a9fbdb4cde9590d9b400ac065a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:48.805770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc787a4-82a4-4a1c-a662-e69ca6f3b71f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:48.807529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 566/1127] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/users/dbe9b2a9fbdb4cde9590d9b400ac065a => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:48.813378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da796213-10ec-44ed-9d93-5adb9c892c3b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:48.839550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:48.917051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da796213-10ec-44ed-9d93-5adb9c892c3b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:48.950886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc64f-58ed-4956-b74f-58f33766e053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:48.951478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc64f-58ed-4956-b74f-58f33766e053 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:48.951756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc64f-58ed-4956-b74f-58f33766e053 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:48.952027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc64f-58ed-4956-b74f-58f33766e053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.017079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfc64f-58ed-4956-b74f-58f33766e053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.018543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfc64f-58ed-4956-b74f-58f33766e053 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.042334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.115778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 562/1128] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:49.122440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc787a4-82a4-4a1c-a662-e69ca6f3b71f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.210013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.222318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc787a4-82a4-4a1c-a662-e69ca6f3b71f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.256400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c2b38-b20f-48c8-8fca-f87b1ad557e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.257058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2b38-b20f-48c8-8fca-f87b1ad557e3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.257340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2b38-b20f-48c8-8fca-f87b1ad557e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.257614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c2b38-b20f-48c8-8fca-f87b1ad557e3 None admin] PATH_INFO: `/v3/projects/3a2dfd9392a74edb87adc1cd427122bb` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.258882 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c2b38-b20f-48c8-8fca-f87b1ad557e3 None admin] RBAC: Authorizing `identity:get_project(project_id=3a2dfd9392a74edb87adc1cd427122bb)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.261045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c2b38-b20f-48c8-8fca-f87b1ad557e3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.262676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 567/1129] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/projects/3a2dfd9392a74edb87adc1cd427122bb => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:49.268892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc64f-58ed-4956-b74f-58f33766e053 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:49.366667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc64f-58ed-4956-b74f-58f33766e053 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:49.390069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:49.401572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4161e-eaa7-4423-bbc7-1ee2d210b0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:49.402169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4161e-eaa7-4423-bbc7-1ee2d210b0de None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:49.402443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4161e-eaa7-4423-bbc7-1ee2d210b0de None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:49.402709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4161e-eaa7-4423-bbc7-1ee2d210b0de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:49.403430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4161e-eaa7-4423-bbc7-1ee2d210b0de None admin] RBAC: Adding query filter params (role.id=79dbf286cf884619af62f33ea75c52bf, scope.project.id=3a2dfd9392a74edb87adc1cd427122bb, user.id=dbe9b2a9fbdb4cde9590d9b400ac065a) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:49.403733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4161e-eaa7-4423-bbc7-1ee2d210b0de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:49.405217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4161e-eaa7-4423-bbc7-1ee2d210b0de None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:49.410802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 563/1130] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:49 2022] GET /identity/v3/role_assignments?user.id=dbe9b2a9fbdb4cde9590d9b400ac065a&scope.project.id=3a2dfd9392a74edb87adc1cd427122bb&role.id=79dbf286cf884619af62f33ea75c52bf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:51.377859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50542d33-437d-40ae-9cef-8ef837e803b4 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.378394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50542d33-437d-40ae-9cef-8ef837e803b4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.378394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50542d33-437d-40ae-9cef-8ef837e803b4 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.379658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 568/1131] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:51.384684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bbe7a-0f10-4b78-a249-b0d2a21ec4e6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.385123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bbe7a-0f10-4b78-a249-b0d2a21ec4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.385225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bbe7a-0f10-4b78-a249-b0d2a21ec4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.403892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a11bbe7a-0f10-4b78-a249-b0d2a21ec4e6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:51.405120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a11bbe7a-0f10-4b78-a249-b0d2a21ec4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:51.471531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 564/1132] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:51.745709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50542d33-437d-40ae-9cef-8ef837e803b4 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:51.847318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50542d33-437d-40ae-9cef-8ef837e803b4 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:51.880012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca1665-979b-4b4d-a774-93e6e47df014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:51.880664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1665-979b-4b4d-a774-93e6e47df014 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:51.880947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1665-979b-4b4d-a774-93e6e47df014 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:51.881220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca1665-979b-4b4d-a774-93e6e47df014 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:51.882330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca1665-979b-4b4d-a774-93e6e47df014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:51.883784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca1665-979b-4b4d-a774-93e6e47df014 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:51.886178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 569/1133] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:51.892520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bbe7a-0f10-4b78-a249-b0d2a21ec4e6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:51.990976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bbe7a-0f10-4b78-a249-b0d2a21ec4e6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:52.008418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:52.028230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306125e-f829-4b3f-aba7-9212aec9adb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:52.028876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306125e-f829-4b3f-aba7-9212aec9adb9 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:52.029157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306125e-f829-4b3f-aba7-9212aec9adb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:52.029434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306125e-f829-4b3f-aba7-9212aec9adb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:52.030099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306125e-f829-4b3f-aba7-9212aec9adb9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:52.031544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306125e-f829-4b3f-aba7-9212aec9adb9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:52.058639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 565/1134] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:05:51 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:54.024468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e2c7-8635-475f-9467-a79a76fed55d None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.025147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e2c7-8635-475f-9467-a79a76fed55d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.025147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e2c7-8635-475f-9467-a79a76fed55d None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.026562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 570/1135] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:54.032335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e152ec-9cc3-4824-b8f5-eee6909e5c30 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.034481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e152ec-9cc3-4824-b8f5-eee6909e5c30 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.034481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e152ec-9cc3-4824-b8f5-eee6909e5c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.052331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-45e152ec-9cc3-4824-b8f5-eee6909e5c30 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:54.053569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-45e152ec-9cc3-4824-b8f5-eee6909e5c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:54.122525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 566/1136] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:05:54.389841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3e2c7-8635-475f-9467-a79a76fed55d None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.414892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.491722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3e2c7-8635-475f-9467-a79a76fed55d None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.525645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d302f72-ac62-4d3a-910c-ddf9e72534ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.526260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d302f72-ac62-4d3a-910c-ddf9e72534ec None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.526541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d302f72-ac62-4d3a-910c-ddf9e72534ec None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.526812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d302f72-ac62-4d3a-910c-ddf9e72534ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.527921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d302f72-ac62-4d3a-910c-ddf9e72534ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.529389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d302f72-ac62-4d3a-910c-ddf9e72534ec None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.530969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 571/1137] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:05:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:54.536751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e152ec-9cc3-4824-b8f5-eee6909e5c30 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:54.631326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:54.634233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e152ec-9cc3-4824-b8f5-eee6909e5c30 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:54.670489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ee7a6-882e-4271-8448-7eaad3b9ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:54.671010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee7a6-882e-4271-8448-7eaad3b9ca73 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:54.671292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee7a6-882e-4271-8448-7eaad3b9ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:54.671565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ee7a6-882e-4271-8448-7eaad3b9ca73 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:54.672225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ee7a6-882e-4271-8448-7eaad3b9ca73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:54.674144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ee7a6-882e-4271-8448-7eaad3b9ca73 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:54.694442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 567/1138] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:05:54 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:05:56.671286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f98a77-9e21-4410-86fe-73fa2c4cedb2 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.671898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f98a77-9e21-4410-86fe-73fa2c4cedb2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.671898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f98a77-9e21-4410-86fe-73fa2c4cedb2 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.673471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 572/1139] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:56.678825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478f859-7bac-461f-92eb-054584ed8829 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:56.679260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478f859-7bac-461f-92eb-054584ed8829 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:56.679353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478f859-7bac-461f-92eb-054584ed8829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:56.699057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f478f859-7bac-461f-92eb-054584ed8829 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:56.700244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f478f859-7bac-461f-92eb-054584ed8829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:56.766817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 568/1140] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:05:57.046310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f98a77-9e21-4410-86fe-73fa2c4cedb2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.140177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.150005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f98a77-9e21-4410-86fe-73fa2c4cedb2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.184405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d14aec-e7d7-4131-b220-8784d6db1caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.185064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14aec-e7d7-4131-b220-8784d6db1caf None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.185343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14aec-e7d7-4131-b220-8784d6db1caf None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.185616 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14aec-e7d7-4131-b220-8784d6db1caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.195399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d14aec-e7d7-4131-b220-8784d6db1caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-29d14aec-e7d7-4131-b220-8784d6db1caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:05:57.200287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:05:57.201419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:05:57.202508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 573/1141] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:05:57.207530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478f859-7bac-461f-92eb-054584ed8829 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.290811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.306436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478f859-7bac-461f-92eb-054584ed8829 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.339667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.340264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.340544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.340843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.341500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:57.341805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.343177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.351098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 569/1142] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:57.357693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d14aec-e7d7-4131-b220-8784d6db1caf None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.457543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d14aec-e7d7-4131-b220-8784d6db1caf None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.491286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c4df5-9018-4aa9-b3d2-5489b7cf8a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.491929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c4df5-9018-4aa9-b3d2-5489b7cf8a44 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.492212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c4df5-9018-4aa9-b3d2-5489b7cf8a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.492494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c4df5-9018-4aa9-b3d2-5489b7cf8a44 None admin] PATH_INFO: `/v3/users/a6a8acc06ccb477a92d7c823a179bf72` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.505172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c4df5-9018-4aa9-b3d2-5489b7cf8a44 None admin] RBAC: Authorizing `identity:get_user(user_id=a6a8acc06ccb477a92d7c823a179bf72)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.506622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c4df5-9018-4aa9-b3d2-5489b7cf8a44 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.508357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 574/1143] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/users/a6a8acc06ccb477a92d7c823a179bf72 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:57.514651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.615909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b9072-a31a-4e3b-b1c1-bafbe9fe7f53 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.648523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3308466-b318-49bf-8dd7-24b3325c26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.649135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3308466-b318-49bf-8dd7-24b3325c26ab None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.650122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3308466-b318-49bf-8dd7-24b3325c26ab None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.650450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3308466-b318-49bf-8dd7-24b3325c26ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.667834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.717391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3308466-b318-49bf-8dd7-24b3325c26ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.718841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3308466-b318-49bf-8dd7-24b3325c26ab None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.814060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 570/1144] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:05:57.820520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c4df5-9018-4aa9-b3d2-5489b7cf8a44 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:57.833461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:57.923034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c4df5-9018-4aa9-b3d2-5489b7cf8a44 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:57.957633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5f2ed-a08f-4718-b8ab-cdb5dec61e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:57.958261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5f2ed-a08f-4718-b8ab-cdb5dec61e93 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:57.958542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5f2ed-a08f-4718-b8ab-cdb5dec61e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:57.958814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5f2ed-a08f-4718-b8ab-cdb5dec61e93 None admin] PATH_INFO: `/v3/projects/581a1e35161b441b9a1a63577e861971` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:57.959932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5f2ed-a08f-4718-b8ab-cdb5dec61e93 None admin] RBAC: Authorizing `identity:get_project(project_id=581a1e35161b441b9a1a63577e861971)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:57.961406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5f2ed-a08f-4718-b8ab-cdb5dec61e93 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:57.963894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 575/1145] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/projects/581a1e35161b441b9a1a63577e861971 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:05:57.970121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3308466-b318-49bf-8dd7-24b3325c26ab None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:58.020168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.073179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3308466-b318-49bf-8dd7-24b3325c26ab None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:58.106415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2dde5-0a37-4a93-98a3-4a93f4010f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:58.107021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2dde5-0a37-4a93-98a3-4a93f4010f1f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.107370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2dde5-0a37-4a93-98a3-4a93f4010f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.107644 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2dde5-0a37-4a93-98a3-4a93f4010f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.108364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2dde5-0a37-4a93-98a3-4a93f4010f1f None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=581a1e35161b441b9a1a63577e861971, user.id=a6a8acc06ccb477a92d7c823a179bf72) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:05:58.108832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2dde5-0a37-4a93-98a3-4a93f4010f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:58.110277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2dde5-0a37-4a93-98a3-4a93f4010f1f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:58.115641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 571/1146] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:05:57 2022] GET /identity/v3/role_assignments?user.id=a6a8acc06ccb477a92d7c823a179bf72&scope.project.id=581a1e35161b441b9a1a63577e861971&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:00.065714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46b82-3f89-42a4-bb40-b0344f302f93 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.066280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46b82-3f89-42a4-bb40-b0344f302f93 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.066280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a46b82-3f89-42a4-bb40-b0344f302f93 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.067593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 576/1147] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:00.072938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33878-1d5a-442e-bea7-9627d41098a5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.073329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33878-1d5a-442e-bea7-9627d41098a5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.073475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33878-1d5a-442e-bea7-9627d41098a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.091794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-bbf33878-1d5a-442e-bea7-9627d41098a5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:00.093009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bbf33878-1d5a-442e-bea7-9627d41098a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:00.158603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 572/1148] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:00.429602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a46b82-3f89-42a4-bb40-b0344f302f93 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.507265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:00.531152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a46b82-3f89-42a4-bb40-b0344f302f93 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.566473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413cd0f-3143-4883-b092-72e11920aca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.567099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413cd0f-3143-4883-b092-72e11920aca1 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.567378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413cd0f-3143-4883-b092-72e11920aca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.567647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413cd0f-3143-4883-b092-72e11920aca1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.578272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413cd0f-3143-4883-b092-72e11920aca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-2413cd0f-3143-4883-b092-72e11920aca1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:00.583077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:00.584167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:06:00.585063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 577/1149] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:00.591085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf33878-1d5a-442e-bea7-9627d41098a5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.634807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:00.693443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf33878-1d5a-442e-bea7-9627d41098a5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.727777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.728382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.728696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.728971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.729635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:00.729936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.731354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.739872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 573/1150] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:00.745899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413cd0f-3143-4883-b092-72e11920aca1 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.845298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413cd0f-3143-4883-b092-72e11920aca1 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.877894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87c8ea-bbb9-4b84-86f6-03394d79d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:00.878509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87c8ea-bbb9-4b84-86f6-03394d79d80c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:00.878790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87c8ea-bbb9-4b84-86f6-03394d79d80c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:00.879057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87c8ea-bbb9-4b84-86f6-03394d79d80c None admin] PATH_INFO: `/v3/users/a6a8acc06ccb477a92d7c823a179bf72` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:00.880160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87c8ea-bbb9-4b84-86f6-03394d79d80c None admin] RBAC: Authorizing `identity:get_user(user_id=a6a8acc06ccb477a92d7c823a179bf72)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:00.881617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87c8ea-bbb9-4b84-86f6-03394d79d80c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:00.883265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 578/1151] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/users/a6a8acc06ccb477a92d7c823a179bf72 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:00.889907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:00.990295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9c344-caad-45c1-95c6-e0e07fa46545 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:00.992642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:01.026460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145bdd5-05c5-4d97-8568-e72ddc73de15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:01.027088 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145bdd5-05c5-4d97-8568-e72ddc73de15 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.027362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145bdd5-05c5-4d97-8568-e72ddc73de15 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.027624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145bdd5-05c5-4d97-8568-e72ddc73de15 None admin] PATH_INFO: `/v3/projects/581a1e35161b441b9a1a63577e861971` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.028774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145bdd5-05c5-4d97-8568-e72ddc73de15 None admin] RBAC: Authorizing `identity:get_project(project_id=581a1e35161b441b9a1a63577e861971)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:01.030300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145bdd5-05c5-4d97-8568-e72ddc73de15 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:01.032006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 574/1152] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:06:00 2022] GET /identity/v3/projects/581a1e35161b441b9a1a63577e861971 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:01.038025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87c8ea-bbb9-4b84-86f6-03394d79d80c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:01.046864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:01.141560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87c8ea-bbb9-4b84-86f6-03394d79d80c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:01.177325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d92951-c423-4060-9460-5a2b9b7efc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:01.177979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92951-c423-4060-9460-5a2b9b7efc26 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.178255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92951-c423-4060-9460-5a2b9b7efc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.178524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92951-c423-4060-9460-5a2b9b7efc26 None admin] PATH_INFO: `/v3/projects/581a1e35161b441b9a1a63577e861971/users/a6a8acc06ccb477a92d7c823a179bf72/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.180427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d92951-c423-4060-9460-5a2b9b7efc26 None admin] RBAC: Authorizing `identity:create_grant(project_id=581a1e35161b441b9a1a63577e861971, user_id=a6a8acc06ccb477a92d7c823a179bf72, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:01.181920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d92951-c423-4060-9460-5a2b9b7efc26 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:01.195034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 579/1153] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:06:01 2022] PUT /identity/v3/projects/581a1e35161b441b9a1a63577e861971/users/a6a8acc06ccb477a92d7c823a179bf72/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:06:03.076983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6345d9-78ee-43b4-810d-f4c203d62861 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.077529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6345d9-78ee-43b4-810d-f4c203d62861 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.077529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6345d9-78ee-43b4-810d-f4c203d62861 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.078858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 575/1154] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:03.084036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fde4a-54cb-4e79-9c46-fe826b97edbc None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.084437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fde4a-54cb-4e79-9c46-fe826b97edbc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.084595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fde4a-54cb-4e79-9c46-fe826b97edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.104672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fc3fde4a-54cb-4e79-9c46-fe826b97edbc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:03.105844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fc3fde4a-54cb-4e79-9c46-fe826b97edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:03.164211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:03.173791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 580/1155] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:03.442148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6345d9-78ee-43b4-810d-f4c203d62861 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:03.551668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6345d9-78ee-43b4-810d-f4c203d62861 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:03.585625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494176-6b46-4e91-b06b-2792bd10207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:03.586244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494176-6b46-4e91-b06b-2792bd10207d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.586525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494176-6b46-4e91-b06b-2792bd10207d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.586986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494176-6b46-4e91-b06b-2792bd10207d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.598014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494176-6b46-4e91-b06b-2792bd10207d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-0e494176-6b46-4e91-b06b-2792bd10207d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:03.603763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:03.605450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:06:03.606868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 576/1156] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:03.611396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fde4a-54cb-4e79-9c46-fe826b97edbc None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:03.712587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fde4a-54cb-4e79-9c46-fe826b97edbc None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:03.748286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:03.748913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.749192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.749469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.750131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:03.750429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:03.751812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:03.760146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 581/1157] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:03.765988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494176-6b46-4e91-b06b-2792bd10207d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:03.797492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:03.867766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494176-6b46-4e91-b06b-2792bd10207d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:03.902065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a7506-12a8-47f0-9dc1-bac979d70d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:03.902698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a7506-12a8-47f0-9dc1-bac979d70d50 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.902979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a7506-12a8-47f0-9dc1-bac979d70d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.903251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a7506-12a8-47f0-9dc1-bac979d70d50 None admin] PATH_INFO: `/v3/users/a6a8acc06ccb477a92d7c823a179bf72` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.904375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a7506-12a8-47f0-9dc1-bac979d70d50 None admin] RBAC: Authorizing `identity:get_user(user_id=a6a8acc06ccb477a92d7c823a179bf72)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:03.905845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a7506-12a8-47f0-9dc1-bac979d70d50 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:03.907439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 577/1158] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:03 2022] GET /identity/v3/users/a6a8acc06ccb477a92d7c823a179bf72 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:03.913405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:03.972608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.015989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5140762-b0b1-4d14-bf10-3bcc9335f046 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.050141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214107e2-8f90-407a-973f-353c64bfcfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.050745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214107e2-8f90-407a-973f-353c64bfcfcb None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.051030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214107e2-8f90-407a-973f-353c64bfcfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.051300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214107e2-8f90-407a-973f-353c64bfcfcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.120560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214107e2-8f90-407a-973f-353c64bfcfcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.122075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214107e2-8f90-407a-973f-353c64bfcfcb None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.179455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.221574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 582/1159] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:04.228215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a7506-12a8-47f0-9dc1-bac979d70d50 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.322101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.326447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a7506-12a8-47f0-9dc1-bac979d70d50 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.361824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660bff4-0bf8-486e-bbbe-1a8f7a5b84f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.362443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660bff4-0bf8-486e-bbbe-1a8f7a5b84f4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.362721 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660bff4-0bf8-486e-bbbe-1a8f7a5b84f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.362990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660bff4-0bf8-486e-bbbe-1a8f7a5b84f4 None admin] PATH_INFO: `/v3/projects/581a1e35161b441b9a1a63577e861971` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.364086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660bff4-0bf8-486e-bbbe-1a8f7a5b84f4 None admin] RBAC: Authorizing `identity:get_project(project_id=581a1e35161b441b9a1a63577e861971)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.365538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660bff4-0bf8-486e-bbbe-1a8f7a5b84f4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.367156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 578/1160] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/projects/581a1e35161b441b9a1a63577e861971 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:04.373923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214107e2-8f90-407a-973f-353c64bfcfcb None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.473405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214107e2-8f90-407a-973f-353c64bfcfcb None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.506408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588f079-a21d-4515-9d10-33dd4e3c65a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.506930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588f079-a21d-4515-9d10-33dd4e3c65a7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.507210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588f079-a21d-4515-9d10-33dd4e3c65a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.507479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588f079-a21d-4515-9d10-33dd4e3c65a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.508194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588f079-a21d-4515-9d10-33dd4e3c65a7 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=581a1e35161b441b9a1a63577e861971, user.id=a6a8acc06ccb477a92d7c823a179bf72) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:04.509050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588f079-a21d-4515-9d10-33dd4e3c65a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:04.510665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588f079-a21d-4515-9d10-33dd4e3c65a7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:04.516279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 583/1161] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/role_assignments?user.id=a6a8acc06ccb477a92d7c823a179bf72&scope.project.id=581a1e35161b441b9a1a63577e861971&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:06.432775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d9252-28af-49ff-be06-7a49c4bdc6f7 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.433329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d9252-28af-49ff-be06-7a49c4bdc6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.433329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d9252-28af-49ff-be06-7a49c4bdc6f7 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.434559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 579/1162] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:06.439589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845042e-75c2-433e-a6ee-3c7b6a39a63a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.439985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845042e-75c2-433e-a6ee-3c7b6a39a63a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.440149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845042e-75c2-433e-a6ee-3c7b6a39a63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.458259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6845042e-75c2-433e-a6ee-3c7b6a39a63a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:06.459459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6845042e-75c2-433e-a6ee-3c7b6a39a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:06.476437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.528376 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 584/1163] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:06.792722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d9252-28af-49ff-be06-7a49c4bdc6f7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.891437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d9252-28af-49ff-be06-7a49c4bdc6f7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.924489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafe6f0-6bdf-4da7-95df-b285551d77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.925112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafe6f0-6bdf-4da7-95df-b285551d77c9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.925382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafe6f0-6bdf-4da7-95df-b285551d77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.925645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafe6f0-6bdf-4da7-95df-b285551d77c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.959305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.992029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafe6f0-6bdf-4da7-95df-b285551d77c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.993636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafe6f0-6bdf-4da7-95df-b285551d77c9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.088430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 580/1164] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:07.094357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845042e-75c2-433e-a6ee-3c7b6a39a63a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.191311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845042e-75c2-433e-a6ee-3c7b6a39a63a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.224276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d19a94-ca31-4f41-a4d8-1c844a1c81e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.224920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d19a94-ca31-4f41-a4d8-1c844a1c81e6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.225200 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d19a94-ca31-4f41-a4d8-1c844a1c81e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.225471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d19a94-ca31-4f41-a4d8-1c844a1c81e6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.232476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d19a94-ca31-4f41-a4d8-1c844a1c81e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.233950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d19a94-ca31-4f41-a4d8-1c844a1c81e6 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-31d19a94-ca31-4f41-a4d8-1c844a1c81e6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:07.245041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:07.246110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:06:07.247133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 585/1165] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:07.252270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafe6f0-6bdf-4da7-95df-b285551d77c9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.316074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:07.348357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafe6f0-6bdf-4da7-95df-b285551d77c9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:07.380876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a57b8-568d-4500-8a20-cd1847cc8f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:07.381462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a57b8-568d-4500-8a20-cd1847cc8f74 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.381739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a57b8-568d-4500-8a20-cd1847cc8f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.382008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a57b8-568d-4500-8a20-cd1847cc8f74 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.382636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a57b8-568d-4500-8a20-cd1847cc8f74 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:07.382932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a57b8-568d-4500-8a20-cd1847cc8f74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:07.385377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a57b8-568d-4500-8a20-cd1847cc8f74 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:07.392263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 581/1166] 10.222.0.95 () {56 vars in 1115 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:09.275585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837018-c9c1-4a0b-8546-3a8526ac85db None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.276115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837018-c9c1-4a0b-8546-3a8526ac85db None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.276115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6837018-c9c1-4a0b-8546-3a8526ac85db None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.277517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 586/1167] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:09.283218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196575-33fe-4d08-844c-c22063716019 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.283646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196575-33fe-4d08-844c-c22063716019 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.283747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196575-33fe-4d08-844c-c22063716019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.302244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d6196575-33fe-4d08-844c-c22063716019 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:09.303426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d6196575-33fe-4d08-844c-c22063716019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:09.370391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 582/1168] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:09.642979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6837018-c9c1-4a0b-8546-3a8526ac85db None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:09.661690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:09.761080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6837018-c9c1-4a0b-8546-3a8526ac85db None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:09.798320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0260c-04c4-4c48-accb-030f9d9cba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:09.798924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0260c-04c4-4c48-accb-030f9d9cba08 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.799202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0260c-04c4-4c48-accb-030f9d9cba08 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.799470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0260c-04c4-4c48-accb-030f9d9cba08 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.800154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0260c-04c4-4c48-accb-030f9d9cba08 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:09.801642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0260c-04c4-4c48-accb-030f9d9cba08 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:09.819996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 587/1169] 10.222.0.95 () {60 vars in 1138 bytes} [Wed Aug 31 19:06:09 2022] POST /identity/v3/domains => generated 267 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:11.803397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3cca-4c37-4e73-a380-e17658cbd953 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.803935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3cca-4c37-4e73-a380-e17658cbd953 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.803935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b3cca-4c37-4e73-a380-e17658cbd953 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.805797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 583/1170] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:11.811245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad685-bb57-4f05-a7a1-f619ee471ab9 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.811847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad685-bb57-4f05-a7a1-f619ee471ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.811847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad685-bb57-4f05-a7a1-f619ee471ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.831250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-7b0ad685-bb57-4f05-a7a1-f619ee471ab9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:11.832423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7b0ad685-bb57-4f05-a7a1-f619ee471ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:11.896582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 588/1171] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:12.166543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3cca-4c37-4e73-a380-e17658cbd953 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.230486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.268417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b3cca-4c37-4e73-a380-e17658cbd953 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.301604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f5b01-3c71-415f-951b-d37588929296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.302215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5b01-3c71-415f-951b-d37588929296 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.302489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5b01-3c71-415f-951b-d37588929296 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.302756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5b01-3c71-415f-951b-d37588929296 None admin] PATH_INFO: `/v3/domains/33f59dd940294cbaa771b3ff8009c3cf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.312869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f5b01-3c71-415f-951b-d37588929296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=33f59dd940294cbaa771b3ff8009c3cf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.314293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f5b01-3c71-415f-951b-d37588929296 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.315850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 584/1172] 10.222.0.95 () {56 vars in 1183 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/domains/33f59dd940294cbaa771b3ff8009c3cf => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:12.321897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad685-bb57-4f05-a7a1-f619ee471ab9 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.324085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.420805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad685-bb57-4f05-a7a1-f619ee471ab9 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.454870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa2c99-e70d-4ef9-b137-f0267a5807aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.455464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2c99-e70d-4ef9-b137-f0267a5807aa None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.455738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2c99-e70d-4ef9-b137-f0267a5807aa None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.456007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2c99-e70d-4ef9-b137-f0267a5807aa None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.456682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa2c99-e70d-4ef9-b137-f0267a5807aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.458120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa2c99-e70d-4ef9-b137-f0267a5807aa None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.485046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 589/1173] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:06:12 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:14.416620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb707b-a1af-458b-8916-63ce2c6c529b None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.417127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb707b-a1af-458b-8916-63ce2c6c529b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.417127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb707b-a1af-458b-8916-63ce2c6c529b None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.418433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 585/1174] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:14.423961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64938e-4f53-482f-8452-61f071b79ddb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.424382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64938e-4f53-482f-8452-61f071b79ddb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.424493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64938e-4f53-482f-8452-61f071b79ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.443164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-1e64938e-4f53-482f-8452-61f071b79ddb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:14.444341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1e64938e-4f53-482f-8452-61f071b79ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:14.492047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:14.510416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 590/1175] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:14.783882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb707b-a1af-458b-8916-63ce2c6c529b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.884967 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb707b-a1af-458b-8916-63ce2c6c529b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.907057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:14.918868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f5e95-2e7c-40bf-ac97-072b1ab74b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.919485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5e95-2e7c-40bf-ac97-072b1ab74b5b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.919767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5e95-2e7c-40bf-ac97-072b1ab74b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.920039 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f5e95-2e7c-40bf-ac97-072b1ab74b5b None admin] PATH_INFO: `/v3/domains/33f59dd940294cbaa771b3ff8009c3cf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.921166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5e95-2e7c-40bf-ac97-072b1ab74b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=33f59dd940294cbaa771b3ff8009c3cf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.922587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f5e95-2e7c-40bf-ac97-072b1ab74b5b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.924407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 586/1176] 10.222.0.95 () {56 vars in 1183 bytes} [Wed Aug 31 19:06:14 2022] GET /identity/v3/domains/33f59dd940294cbaa771b3ff8009c3cf => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:14.930547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64938e-4f53-482f-8452-61f071b79ddb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:15.027024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64938e-4f53-482f-8452-61f071b79ddb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:15.060431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0dade-a774-478e-8ace-13560d993fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:15.060991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0dade-a774-478e-8ace-13560d993fbb None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:15.061275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0dade-a774-478e-8ace-13560d993fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:15.061549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0dade-a774-478e-8ace-13560d993fbb None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:15.062198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0dade-a774-478e-8ace-13560d993fbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:15.063651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0dade-a774-478e-8ace-13560d993fbb None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:15.086017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 591/1177] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:06:14 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:06:17.088731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60039c-09e8-40cb-918c-fdc089c41905 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.089304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60039c-09e8-40cb-918c-fdc089c41905 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.089304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60039c-09e8-40cb-918c-fdc089c41905 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.090591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 587/1178] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:17.095653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193eb19-17f4-4347-a72e-2c03067a5fe8 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.096052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193eb19-17f4-4347-a72e-2c03067a5fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.096216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193eb19-17f4-4347-a72e-2c03067a5fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.114970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-1193eb19-17f4-4347-a72e-2c03067a5fe8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:17.116169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1193eb19-17f4-4347-a72e-2c03067a5fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:17.154175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:17.187386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 592/1179] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:17.460843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60039c-09e8-40cb-918c-fdc089c41905 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.569127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60039c-09e8-40cb-918c-fdc089c41905 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.602783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995e9e-928c-4421-b8fe-2e8b74befd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.603391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995e9e-928c-4421-b8fe-2e8b74befd79 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.603668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995e9e-928c-4421-b8fe-2e8b74befd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.603974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e995e9e-928c-4421-b8fe-2e8b74befd79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.615192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e995e9e-928c-4421-b8fe-2e8b74befd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3e995e9e-928c-4421-b8fe-2e8b74befd79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:17.620740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:17.621903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:06:17.622962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 588/1180] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:17.627945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193eb19-17f4-4347-a72e-2c03067a5fe8 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.726504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193eb19-17f4-4347-a72e-2c03067a5fe8 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.762020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.762625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.762907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.763178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.763840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:17.764141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.765557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.773586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 593/1181] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:17.779697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995e9e-928c-4421-b8fe-2e8b74befd79 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.829391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:17.879350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e995e9e-928c-4421-b8fe-2e8b74befd79 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:17.911309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c641b-2ab4-4577-ab8d-4eabee409a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:17.911926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c641b-2ab4-4577-ab8d-4eabee409a01 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:17.912209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c641b-2ab4-4577-ab8d-4eabee409a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:17.912482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c641b-2ab4-4577-ab8d-4eabee409a01 None admin] PATH_INFO: `/v3/users/32a41d5e0ec34baeac19fc3016e56b0a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:17.925968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c641b-2ab4-4577-ab8d-4eabee409a01 None admin] RBAC: Authorizing `identity:get_user(user_id=32a41d5e0ec34baeac19fc3016e56b0a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:17.927417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c641b-2ab4-4577-ab8d-4eabee409a01 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:17.929456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 589/1182] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/users/32a41d5e0ec34baeac19fc3016e56b0a => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:17.936000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:17.967211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.035787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e041d-6d29-403c-b002-0714f4ae833c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.069330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e5d72-d533-4bbe-97b8-77886d7e7b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.069942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e5d72-d533-4bbe-97b8-77886d7e7b5e None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.070225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e5d72-d533-4bbe-97b8-77886d7e7b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.070499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e5d72-d533-4bbe-97b8-77886d7e7b5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.137343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e5d72-d533-4bbe-97b8-77886d7e7b5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.138804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e5d72-d533-4bbe-97b8-77886d7e7b5e None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.167698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.236920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 594/1183] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:18.243038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c641b-2ab4-4577-ab8d-4eabee409a01 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.339676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c641b-2ab4-4577-ab8d-4eabee409a01 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.348674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.373831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092fced-1d0b-436f-84df-6b6dba5c6ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.374448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092fced-1d0b-436f-84df-6b6dba5c6ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.374726 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092fced-1d0b-436f-84df-6b6dba5c6ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.374997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092fced-1d0b-436f-84df-6b6dba5c6ca6 None admin] PATH_INFO: `/v3/projects/07090937f0c046a7a41d3c3a8921b1f6` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.376090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092fced-1d0b-436f-84df-6b6dba5c6ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=07090937f0c046a7a41d3c3a8921b1f6)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.377558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092fced-1d0b-436f-84df-6b6dba5c6ca6 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.379182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 590/1184] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/projects/07090937f0c046a7a41d3c3a8921b1f6 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:18.385510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e5d72-d533-4bbe-97b8-77886d7e7b5e None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:18.484552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e5d72-d533-4bbe-97b8-77886d7e7b5e None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:18.512541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:18.517798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd3144-4034-49d8-934f-e2f87501a9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:18.518392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3144-4034-49d8-934f-e2f87501a9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:18.518667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3144-4034-49d8-934f-e2f87501a9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:18.518934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3144-4034-49d8-934f-e2f87501a9f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:18.519647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd3144-4034-49d8-934f-e2f87501a9f0 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=07090937f0c046a7a41d3c3a8921b1f6, user.id=32a41d5e0ec34baeac19fc3016e56b0a) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:18.519948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd3144-4034-49d8-934f-e2f87501a9f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:18.521430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd3144-4034-49d8-934f-e2f87501a9f0 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:18.527426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 595/1185] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:18 2022] GET /identity/v3/role_assignments?user.id=32a41d5e0ec34baeac19fc3016e56b0a&scope.project.id=07090937f0c046a7a41d3c3a8921b1f6&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:20.405558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe57f18-0f88-4a6f-9463-4179b9138a47 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:20.406092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe57f18-0f88-4a6f-9463-4179b9138a47 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:20.406092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe57f18-0f88-4a6f-9463-4179b9138a47 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:20.407373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 591/1186] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:20.412622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc73d1e-af18-489f-b110-4e59d7a6a787 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:20.413051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc73d1e-af18-489f-b110-4e59d7a6a787 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:20.413170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc73d1e-af18-489f-b110-4e59d7a6a787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:20.432403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-5fc73d1e-af18-489f-b110-4e59d7a6a787 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:20.433630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5fc73d1e-af18-489f-b110-4e59d7a6a787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:20.502268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 596/1187] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:20.767879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe57f18-0f88-4a6f-9463-4179b9138a47 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:20.867144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe57f18-0f88-4a6f-9463-4179b9138a47 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:20.901347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680564d-d002-46c8-9fa6-1814a28ee9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:20.901966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680564d-d002-46c8-9fa6-1814a28ee9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:20.902248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680564d-d002-46c8-9fa6-1814a28ee9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:20.902523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680564d-d002-46c8-9fa6-1814a28ee9f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:20.912206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680564d-d002-46c8-9fa6-1814a28ee9f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a680564d-d002-46c8-9fa6-1814a28ee9f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:20.918049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:20.919185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:06:20.921777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 592/1188] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:20.925519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc73d1e-af18-489f-b110-4e59d7a6a787 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:21.028952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc73d1e-af18-489f-b110-4e59d7a6a787 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:21.057075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:21.065870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:21.066477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.066754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.067021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.067683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:21.067983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:21.069396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:21.077937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 597/1189] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:21.083882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680564d-d002-46c8-9fa6-1814a28ee9f3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:21.125479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:21.194457 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680564d-d002-46c8-9fa6-1814a28ee9f3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:21.228763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e2611-9044-436f-91ed-c68cf5e979ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:21.229385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e2611-9044-436f-91ed-c68cf5e979ae None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.229664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e2611-9044-436f-91ed-c68cf5e979ae None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.229935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e2611-9044-436f-91ed-c68cf5e979ae None admin] PATH_INFO: `/v3/users/32a41d5e0ec34baeac19fc3016e56b0a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.231044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e2611-9044-436f-91ed-c68cf5e979ae None admin] RBAC: Authorizing `identity:get_user(user_id=32a41d5e0ec34baeac19fc3016e56b0a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:21.232470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e2611-9044-436f-91ed-c68cf5e979ae None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:21.234135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 593/1190] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:21 2022] GET /identity/v3/users/32a41d5e0ec34baeac19fc3016e56b0a => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:21.240221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:21.339396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb98989-9e01-499f-8ba2-58632c141673 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:21.372339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3c6f8-631a-44ad-bacf-e011f29d5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:21.372991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3c6f8-631a-44ad-bacf-e011f29d5294 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.373269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3c6f8-631a-44ad-bacf-e011f29d5294 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.373536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3c6f8-631a-44ad-bacf-e011f29d5294 None admin] PATH_INFO: `/v3/projects/07090937f0c046a7a41d3c3a8921b1f6` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.374719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3c6f8-631a-44ad-bacf-e011f29d5294 None admin] RBAC: Authorizing `identity:get_project(project_id=07090937f0c046a7a41d3c3a8921b1f6)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:21.376165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3c6f8-631a-44ad-bacf-e011f29d5294 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:21.377819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 598/1191] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:06:21 2022] GET /identity/v3/projects/07090937f0c046a7a41d3c3a8921b1f6 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:21.384016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e2611-9044-436f-91ed-c68cf5e979ae None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:21.482007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e2611-9044-436f-91ed-c68cf5e979ae None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:21.484260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:21.514925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487924a-d0ec-444a-9e06-1b51f5044844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:21.515573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487924a-d0ec-444a-9e06-1b51f5044844 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:21.515854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487924a-d0ec-444a-9e06-1b51f5044844 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:21.516122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487924a-d0ec-444a-9e06-1b51f5044844 None admin] PATH_INFO: `/v3/projects/07090937f0c046a7a41d3c3a8921b1f6/users/32a41d5e0ec34baeac19fc3016e56b0a/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:21.518059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487924a-d0ec-444a-9e06-1b51f5044844 None admin] RBAC: Authorizing `identity:create_grant(project_id=07090937f0c046a7a41d3c3a8921b1f6, user_id=32a41d5e0ec34baeac19fc3016e56b0a, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:21.519492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487924a-d0ec-444a-9e06-1b51f5044844 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:21.532363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 594/1192] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:06:21 2022] PUT /identity/v3/projects/07090937f0c046a7a41d3c3a8921b1f6/users/32a41d5e0ec34baeac19fc3016e56b0a/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:06:23.435398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084addcc-3e3e-457c-ba1d-89817d8d4526 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.435912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084addcc-3e3e-457c-ba1d-89817d8d4526 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.435912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084addcc-3e3e-457c-ba1d-89817d8d4526 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.437232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 599/1193] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:23.442311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867a2d-fc9b-4d3c-b4c3-3f1f55b6a274 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.442711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867a2d-fc9b-4d3c-b4c3-3f1f55b6a274 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.442856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867a2d-fc9b-4d3c-b4c3-3f1f55b6a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.460583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5c867a2d-fc9b-4d3c-b4c3-3f1f55b6a274 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:23.462309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5c867a2d-fc9b-4d3c-b4c3-3f1f55b6a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:23.535044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 595/1194] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:06:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:06:23.812123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084addcc-3e3e-457c-ba1d-89817d8d4526 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:23.874068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:23.921672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084addcc-3e3e-457c-ba1d-89817d8d4526 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:23.957460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af938466-3d88-46d2-9eb9-8e0818617f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:23.958088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af938466-3d88-46d2-9eb9-8e0818617f8f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:23.958367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af938466-3d88-46d2-9eb9-8e0818617f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:23.958639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af938466-3d88-46d2-9eb9-8e0818617f8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:23.969309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af938466-3d88-46d2-9eb9-8e0818617f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-af938466-3d88-46d2-9eb9-8e0818617f8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:06:23.974143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:06:23.975292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:06:23.976319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 600/1195] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:06:23.984777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c867a2d-fc9b-4d3c-b4c3-3f1f55b6a274 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.076060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.083167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c867a2d-fc9b-4d3c-b4c3-3f1f55b6a274 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.116547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.117180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.118044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.118315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.118985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:24.119316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.120762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.129437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 596/1196] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:24.135982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af938466-3d88-46d2-9eb9-8e0818617f8f None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.238067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af938466-3d88-46d2-9eb9-8e0818617f8f None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.254504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.272897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e617a-d493-4656-b1fa-249b094a57bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.274543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e617a-d493-4656-b1fa-249b094a57bd None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.274836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e617a-d493-4656-b1fa-249b094a57bd None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.275109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e617a-d493-4656-b1fa-249b094a57bd None admin] PATH_INFO: `/v3/users/32a41d5e0ec34baeac19fc3016e56b0a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.276253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e617a-d493-4656-b1fa-249b094a57bd None admin] RBAC: Authorizing `identity:get_user(user_id=32a41d5e0ec34baeac19fc3016e56b0a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.277733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e617a-d493-4656-b1fa-249b094a57bd None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.279319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 601/1197] 10.222.0.95 () {56 vars in 1177 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/users/32a41d5e0ec34baeac19fc3016e56b0a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:24.285340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.382454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4c477-1945-42d9-9458-b6a2b2468edc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.418254 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50db1d-5479-480e-a83c-df505e4e9987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.418872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50db1d-5479-480e-a83c-df505e4e9987 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.419156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50db1d-5479-480e-a83c-df505e4e9987 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.419422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50db1d-5479-480e-a83c-df505e4e9987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.446592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.492809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50db1d-5479-480e-a83c-df505e4e9987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.494285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50db1d-5479-480e-a83c-df505e4e9987 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.593232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 597/1198] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:06:24.599308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e617a-d493-4656-b1fa-249b094a57bd None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.698118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e617a-d493-4656-b1fa-249b094a57bd None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.730819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a9d9b-1b52-485e-b14e-dc7359326058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.731428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9d9b-1b52-485e-b14e-dc7359326058 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.731706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9d9b-1b52-485e-b14e-dc7359326058 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.731974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9d9b-1b52-485e-b14e-dc7359326058 None admin] PATH_INFO: `/v3/projects/07090937f0c046a7a41d3c3a8921b1f6` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.733101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a9d9b-1b52-485e-b14e-dc7359326058 None admin] RBAC: Authorizing `identity:get_project(project_id=07090937f0c046a7a41d3c3a8921b1f6)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.734540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a9d9b-1b52-485e-b14e-dc7359326058 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.736152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 602/1199] 10.222.0.95 () {56 vars in 1186 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/projects/07090937f0c046a7a41d3c3a8921b1f6 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:24.743325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50db1d-5479-480e-a83c-df505e4e9987 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:24.796695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.839232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50db1d-5479-480e-a83c-df505e4e9987 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:24.873139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1c9c5-12a8-46c0-8633-1b1deaadd94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:24.873732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c9c5-12a8-46c0-8633-1b1deaadd94a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.874011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c9c5-12a8-46c0-8633-1b1deaadd94a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.874282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c9c5-12a8-46c0-8633-1b1deaadd94a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.874996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1c9c5-12a8-46c0-8633-1b1deaadd94a None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=07090937f0c046a7a41d3c3a8921b1f6, user.id=32a41d5e0ec34baeac19fc3016e56b0a) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:06:24.875302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1c9c5-12a8-46c0-8633-1b1deaadd94a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:24.876752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1c9c5-12a8-46c0-8633-1b1deaadd94a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:24.882362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 598/1200] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:06:24 2022] GET /identity/v3/role_assignments?user.id=32a41d5e0ec34baeac19fc3016e56b0a&scope.project.id=07090937f0c046a7a41d3c3a8921b1f6&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:06:34.726151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=74 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:34.879743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=40 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:21.623202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870026c-b42b-4fc2-9795-9465cd0dd6d4 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:21.623202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870026c-b42b-4fc2-9795-9465cd0dd6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:21.623202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870026c-b42b-4fc2-9795-9465cd0dd6d4 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:21.623202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 603/1201] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:21.629413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152d40-f272-4e3e-b187-0eeda0e5a387 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:21.630033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152d40-f272-4e3e-b187-0eeda0e5a387 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:21.630033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152d40-f272-4e3e-b187-0eeda0e5a387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:21.654216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ed152d40-f272-4e3e-b187-0eeda0e5a387 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:21.655479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ed152d40-f272-4e3e-b187-0eeda0e5a387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:21.756464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 599/1202] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:22.055946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870026c-b42b-4fc2-9795-9465cd0dd6d4 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:22.214617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870026c-b42b-4fc2-9795-9465cd0dd6d4 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:22.266257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:22.266987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:22.267320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:22.267597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:22.286462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:22.288546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:22.306038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:22.307163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:22.310935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:22.310935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:22.310935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:22.310935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 604/1203] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:22.314776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed152d40-f272-4e3e-b187-0eeda0e5a387 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:22.466783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed152d40-f272-4e3e-b187-0eeda0e5a387 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:22.517549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:22.526026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9514b9-251f-4d47-8a0b-3fc9941fefb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:22.526733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9514b9-251f-4d47-8a0b-3fc9941fefb2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:22.528100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9514b9-251f-4d47-8a0b-3fc9941fefb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:22.528667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9514b9-251f-4d47-8a0b-3fc9941fefb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:22.529609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9514b9-251f-4d47-8a0b-3fc9941fefb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:22.530215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9514b9-251f-4d47-8a0b-3fc9941fefb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:22.532278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9514b9-251f-4d47-8a0b-3fc9941fefb2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:22.553298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 600/1204] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:22.557446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:22.620994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:22.686674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb0e8e-3fe4-4ee9-9855-c023cb661835 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:22.736326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4e1d9-452e-4313-974f-f92a93022439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:22.739020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e1d9-452e-4313-974f-f92a93022439 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:22.739020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e1d9-452e-4313-974f-f92a93022439 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:22.739020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4e1d9-452e-4313-974f-f92a93022439 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:22.739428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4e1d9-452e-4313-974f-f92a93022439 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:22.741624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4e1d9-452e-4313-974f-f92a93022439 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:22.771356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 605/1205] 10.222.0.95 () {60 vars in 1132 bytes} [Wed Aug 31 19:07:22 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:07:24.967078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9658-10b9-4e3b-99af-67e7a96b0bf0 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:24.967665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9658-10b9-4e3b-99af-67e7a96b0bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:24.967665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9658-10b9-4e3b-99af-67e7a96b0bf0 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:24.969404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 601/1206] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:24.975994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b24db1-ee31-4389-b45e-5f550024c49c None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:24.976776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b24db1-ee31-4389-b45e-5f550024c49c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:24.976776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b24db1-ee31-4389-b45e-5f550024c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:25.001660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-29b24db1-ee31-4389-b45e-5f550024c49c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:25.003077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-29b24db1-ee31-4389-b45e-5f550024c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:25.082743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 606/1207] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:07:25.361060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c9658-10b9-4e3b-99af-67e7a96b0bf0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:25.493089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c9658-10b9-4e3b-99af-67e7a96b0bf0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:25.542506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466800-128b-458c-9d96-d517401509f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:25.543420 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466800-128b-458c-9d96-d517401509f6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:25.543718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466800-128b-458c-9d96-d517401509f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:25.543964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466800-128b-458c-9d96-d517401509f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:25.557434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2466800-128b-458c-9d96-d517401509f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b2466800-128b-458c-9d96-d517401509f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:25.562785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:25.564026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:25.565505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 602/1208] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:07:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:25.570509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b24db1-ee31-4389-b45e-5f550024c49c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:25.623611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:25.744767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b24db1-ee31-4389-b45e-5f550024c49c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:25.807901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:25.809117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:25.809593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:25.810157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:25.811451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:25.812149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:25.814717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:25.836624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 607/1209] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:07:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:25.840410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466800-128b-458c-9d96-d517401509f6 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:25.903765 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:25.981241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466800-128b-458c-9d96-d517401509f6 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:26.028058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:26.028748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:26.029035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:26.029315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:26.042320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:26.044033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:26.061132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:26.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:26.064919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:26.064919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:26.064919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:26.064919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 603/1210] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:26.069824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:26.203444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:26.206819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d4136-3e7d-432b-afde-d23fd41eaf8a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:26.258741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:26.259366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:26.259648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:26.259926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:26.260648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:26.260957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:26.262351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:26.285680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 608/1211] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:26.287032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:26.420099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58991-6135-407c-84ca-43644cc73d7c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:26.422076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:26.463217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:26.463860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:26.464142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:26.464478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:26.473440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:26.475062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:07:26.489045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:26.490081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:26.491676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 604/1212] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:26.499750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:26.632791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b79bf3-918e-41a7-9bc8-92c8ee0ceac4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:26.682430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:26.683071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:26.683388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:26.683657 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:26.684435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:26.684774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:26.686335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:26.705683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 609/1213] 10.222.0.95 () {56 vars in 1143 bytes} [Wed Aug 31 19:07:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:26.714289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:26.861569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeaefd-9aa3-4b1c-8039-a1442e8443c3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:26.905080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d174c-208f-432f-a2dc-c92bd7765b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:26.905710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d174c-208f-432f-a2dc-c92bd7765b32 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:26.905996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d174c-208f-432f-a2dc-c92bd7765b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:26.906366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d174c-208f-432f-a2dc-c92bd7765b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:26.943322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:26.990247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d174c-208f-432f-a2dc-c92bd7765b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:26.992097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d174c-208f-432f-a2dc-c92bd7765b32 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:27.114718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 605/1214] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:07:27.119711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:27.157648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:27.261416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728126c3-6652-4a0c-ab67-856928f30f60 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:27.315702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:27.316522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:27.316825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:27.317143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:27.332838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:27.334486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:27.353560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:27.354822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:27.356138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:27.356138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:27.356138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:27.360780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 610/1215] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:07:27.364418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d174c-208f-432f-a2dc-c92bd7765b32 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:27.452996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:27.505329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d174c-208f-432f-a2dc-c92bd7765b32 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:27.549045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:27.549663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:27.549942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:27.550214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:27.550895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:27.551595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:27.552649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:27.573934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 606/1216] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:27.575385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:27.670823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:27.708429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3d3f5-a132-4ca8-b5ca-14581eceaf45 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:27.750856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:27.751504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:27.751815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:27.752147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:27.762402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:27.764041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:07:27.781154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:27.782573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:27.783965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:07:27.783965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:07:27.783965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:07:27.783965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:07:27.783965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:07:27.783965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:07:27.783965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:27.783965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:27.787390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 611/1217] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:07:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:27.789113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:27.917089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7c2f0-de83-4b4c-bf87-3ded657c7389 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:27.946118 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:27.959522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382566e8-1c0e-4c98-96ec-df78b891bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:27.960257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382566e8-1c0e-4c98-96ec-df78b891bf92 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:27.960545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382566e8-1c0e-4c98-96ec-df78b891bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:27.960861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382566e8-1c0e-4c98-96ec-df78b891bf92 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:27.962380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382566e8-1c0e-4c98-96ec-df78b891bf92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:27.962380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382566e8-1c0e-4c98-96ec-df78b891bf92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:27.963283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382566e8-1c0e-4c98-96ec-df78b891bf92 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:27.979127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 607/1218] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:07:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:27.987852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:28.130389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d8c1b-c018-4588-82cd-62f172f567af None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:28.165861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:28.173448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34403203-8b14-48e9-9ae3-e7e6c3db28e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:28.174035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34403203-8b14-48e9-9ae3-e7e6c3db28e0 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:28.174318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34403203-8b14-48e9-9ae3-e7e6c3db28e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:28.176712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34403203-8b14-48e9-9ae3-e7e6c3db28e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:28.176712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34403203-8b14-48e9-9ae3-e7e6c3db28e0 None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=a39f8675dcfa4988af3a402b32984b84) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:28.176712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34403203-8b14-48e9-9ae3-e7e6c3db28e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:28.177473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34403203-8b14-48e9-9ae3-e7e6c3db28e0 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:28.196619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 612/1219] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:27 2022] GET /identity/v3/role_assignments?user.id=a39f8675dcfa4988af3a402b32984b84&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:30.197250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9ccd2-9a9d-4756-b087-88fb96cc33e9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:30.197787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9ccd2-9a9d-4756-b087-88fb96cc33e9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:30.197787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9ccd2-9a9d-4756-b087-88fb96cc33e9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:30.199574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 608/1220] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:07:30.206952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286ba0e-0fa9-4e80-b262-006ddc8e9545 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:30.208795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286ba0e-0fa9-4e80-b262-006ddc8e9545 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:30.208795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286ba0e-0fa9-4e80-b262-006ddc8e9545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:30.230210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-0286ba0e-0fa9-4e80-b262-006ddc8e9545 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:30.231434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0286ba0e-0fa9-4e80-b262-006ddc8e9545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:30.329429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 613/1221] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:30.603779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9ccd2-9a9d-4756-b087-88fb96cc33e9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:30.747787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9ccd2-9a9d-4756-b087-88fb96cc33e9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:30.788901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eee3bc-69fc-448f-ae59-1862df3f79da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:30.789526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eee3bc-69fc-448f-ae59-1862df3f79da None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:30.789812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eee3bc-69fc-448f-ae59-1862df3f79da None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:30.790094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eee3bc-69fc-448f-ae59-1862df3f79da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:30.801748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eee3bc-69fc-448f-ae59-1862df3f79da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-85eee3bc-69fc-448f-ae59-1862df3f79da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:30.808090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:30.809578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:30.810799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 609/1222] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:07:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:30.815607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286ba0e-0fa9-4e80-b262-006ddc8e9545 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:30.932910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286ba0e-0fa9-4e80-b262-006ddc8e9545 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:30.960682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:30.975538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:30.976298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:30.976585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:30.976867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:30.977564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] RBAC: Adding query filter params (name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:30.977916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:30.979309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:30.990139 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 614/1223] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:30.997334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eee3bc-69fc-448f-ae59-1862df3f79da None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:31.069380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:31.141462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eee3bc-69fc-448f-ae59-1862df3f79da None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:31.189131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:31.190192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:31.190482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:31.190761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:31.205614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:31.207125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:31.223592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:31.225717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:31.227002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:31.227002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:31.227002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:31.227002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 610/1224] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:31.232098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:31.358570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea56fba-c73e-45c5-9ec1-3c26fa32f933 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:31.403143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:31.403862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:31.404138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:31.404414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:31.405089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:31.405391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:31.406782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:31.420062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 615/1225] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:31.426860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:31.537017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:31.547480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14a2e6-a7d1-4fa8-a5fd-bcd49dd3a5c2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:31.593024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:31.593673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:31.594086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:31.594445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:31.604269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:31.605977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:07:31.620701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:31.628690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:31.629978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 611/1226] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:07:31.630585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:31.682986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:31.751572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aec68-d8c9-4cfa-8c9f-c9893c57bb4b None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:31.791512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:31.792068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:31.792345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:31.792681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:31.793595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:31.793906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:31.795374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:31.818201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 616/1227] 10.222.0.95 () {56 vars in 1143 bytes} [Wed Aug 31 19:07:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:31.824757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:31.940689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e8a91-813f-441a-8624-feed3cc690f4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:31.982167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:31.982801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:31.983081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:31.983354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:31.996142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:31.997705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:32.013788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:32.015332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:32.018798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:32.018798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:32.018798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:32.018798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 612/1228] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:32.021411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:32.144135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff63a-af17-4bc0-a2e7-82348d7f00d7 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:32.146873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:32.183180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:32.183816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:32.184094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:32.184368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:32.185158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:32.185463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:32.186928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:32.200058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 617/1229] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:32.206261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:32.218071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:32.332288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bd0bf-a2f2-488e-b2d6-2da76267e942 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:32.375808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:32.377433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:32.377433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:32.378175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:32.389386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:32.391442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:07:32.411056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:32.413148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:32.414672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 613/1230] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:07:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:32.419987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:32.533716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239bf6e-8cb4-4564-aa9a-dd0949832e30 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:32.570553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77a807-b5f1-4820-a178-1c70e1a95a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:32.571174 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77a807-b5f1-4820-a178-1c70e1a95a93 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:32.571459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77a807-b5f1-4820-a178-1c70e1a95a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:32.571727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77a807-b5f1-4820-a178-1c70e1a95a93 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:32.572430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77a807-b5f1-4820-a178-1c70e1a95a93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:32.572759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77a807-b5f1-4820-a178-1c70e1a95a93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:32.574264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77a807-b5f1-4820-a178-1c70e1a95a93 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:32.589933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 618/1231] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:07:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:32.597083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:32.692489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:32.734814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bbcfd-24e4-4f52-a441-852c05a52bb5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:32.790487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a933c-a965-4ffa-b90b-06f7df31534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:32.791310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a933c-a965-4ffa-b90b-06f7df31534e None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:32.791563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a933c-a965-4ffa-b90b-06f7df31534e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:32.791835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a933c-a965-4ffa-b90b-06f7df31534e None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/a39f8675dcfa4988af3a402b32984b84/roles/c49d2fa980524a9f95d78b7fff997d1c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:32.810637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a933c-a965-4ffa-b90b-06f7df31534e None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=a39f8675dcfa4988af3a402b32984b84, role_id=c49d2fa980524a9f95d78b7fff997d1c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:32.812760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a933c-a965-4ffa-b90b-06f7df31534e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:32.829227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 614/1232] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:07:32 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/a39f8675dcfa4988af3a402b32984b84/roles/c49d2fa980524a9f95d78b7fff997d1c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:07:34.892516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3b0f7-ba18-4a49-804d-654382dba308 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:34.893310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3b0f7-ba18-4a49-804d-654382dba308 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:34.893310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3b0f7-ba18-4a49-804d-654382dba308 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:34.896621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 619/1233] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:34.902501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c54b0-efc3-4832-999f-4f2b6ef50596 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:34.902501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c54b0-efc3-4832-999f-4f2b6ef50596 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:34.903115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c54b0-efc3-4832-999f-4f2b6ef50596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:34.925693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c32c54b0-efc3-4832-999f-4f2b6ef50596 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:34.927183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c32c54b0-efc3-4832-999f-4f2b6ef50596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:35.016662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 615/1234] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:35.331343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3b0f7-ba18-4a49-804d-654382dba308 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:35.363627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:35.462077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3b0f7-ba18-4a49-804d-654382dba308 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:35.524001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4b7bc-e141-453b-9eb3-d3eac98952e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:35.525265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4b7bc-e141-453b-9eb3-d3eac98952e6 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:35.525736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4b7bc-e141-453b-9eb3-d3eac98952e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:35.526186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4b7bc-e141-453b-9eb3-d3eac98952e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:35.545224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4b7bc-e141-453b-9eb3-d3eac98952e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-bcd4b7bc-e141-453b-9eb3-d3eac98952e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:35.552815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:35.554336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:35.555273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 620/1235] 10.222.0.95 () {56 vars in 1102 bytes} [Wed Aug 31 19:07:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:35.562572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c54b0-efc3-4832-999f-4f2b6ef50596 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:35.618418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:35.701176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c54b0-efc3-4832-999f-4f2b6ef50596 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:35.752723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:35.753447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:35.753764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:35.754126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:35.755120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:35.755566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:35.757710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:35.768676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 616/1236] 10.222.0.95 () {56 vars in 1103 bytes} [Wed Aug 31 19:07:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:35.774569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4b7bc-e141-453b-9eb3-d3eac98952e6 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:35.891763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:35.914627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4b7bc-e141-453b-9eb3-d3eac98952e6 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:35.965219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:35.965899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:35.966269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:35.966665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:35.980247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:35.982173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:35.999013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:36.000146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:36.001095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:36.001095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:36.001095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:36.001095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 621/1237] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:07:36.008079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:36.150721 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207411f0-0dfb-4b6f-9519-b99a0a99b8d4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:36.166664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:36.202847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:36.203466 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:36.203748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:36.204020 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:36.204848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:36.205154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:36.206528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:36.223218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 617/1238] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:36.231170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:36.378479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d54524-5d09-4300-94ff-4e4761a8edc9 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:36.420747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:36.421710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:36.422033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:36.422321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:36.425706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:36.433947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:36.435851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:07:36.453890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:36.455911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:36.457765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 622/1239] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:36.461196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:36.593862 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b970b9f-300b-43aa-bb92-36dc0731548d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:36.633214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:36.633839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:36.634145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:36.634381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:36.635123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:36.635423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:36.636882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:36.665684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 618/1240] 10.222.0.95 () {56 vars in 1143 bytes} [Wed Aug 31 19:07:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:36.673273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:36.798994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41534e16-016e-4d4e-aa66-28d0d5fc73cb None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:36.836772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312397-ccad-4fb8-8fa8-56b744d4b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:36.837374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312397-ccad-4fb8-8fa8-56b744d4b067 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:36.837655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312397-ccad-4fb8-8fa8-56b744d4b067 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:36.837926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312397-ccad-4fb8-8fa8-56b744d4b067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:36.904781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:36.912289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312397-ccad-4fb8-8fa8-56b744d4b067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:36.914600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312397-ccad-4fb8-8fa8-56b744d4b067 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:37.032828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 623/1241] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 19:07:37.038882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:37.073825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:37.165232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c0f89-ed0b-4c37-851a-9b5c67a923cc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:37.214770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:37.215781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:37.216417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:37.216972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:37.238276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:37.240322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:37.261297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:37.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:37.263989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:37.263989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:37.263989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:37.267385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 619/1242] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:37.268411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312397-ccad-4fb8-8fa8-56b744d4b067 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:37.384735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:37.391604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312397-ccad-4fb8-8fa8-56b744d4b067 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:37.436858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:37.437667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:37.438084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:37.438529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:37.439569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:37.440317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:37.442321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:37.457281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 624/1243] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:37.464460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:37.563143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:37.592028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50c7d5-b20a-4e69-b11d-f2e12726f287 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:37.635725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:37.636355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:37.636658 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:37.637006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:37.646477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:37.649705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:07:37.670578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:37.672036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:37.673271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 620/1244] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:07:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:07:37.679598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:37.818906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48a3e3-8de9-480e-a8bb-c81a893cf970 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:37.870118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1929a03-44e6-45f2-b292-05ef4210471d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:37.870882 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1929a03-44e6-45f2-b292-05ef4210471d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:37.871192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1929a03-44e6-45f2-b292-05ef4210471d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:37.871460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1929a03-44e6-45f2-b292-05ef4210471d None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:37.872306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1929a03-44e6-45f2-b292-05ef4210471d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:37.872853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1929a03-44e6-45f2-b292-05ef4210471d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:37.874779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1929a03-44e6-45f2-b292-05ef4210471d None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:37.896669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 625/1245] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:37.897965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:38.019461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c322d-c005-4498-8fd8-5ab0412436ce None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:38.063087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6213-06ce-41f5-a642-6ba7b46e9375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:38.063812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6213-06ce-41f5-a642-6ba7b46e9375 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:38.064087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6213-06ce-41f5-a642-6ba7b46e9375 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:38.064388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6213-06ce-41f5-a642-6ba7b46e9375 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:38.065293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c6213-06ce-41f5-a642-6ba7b46e9375 None admin] RBAC: Adding query filter params (role.id=c49d2fa980524a9f95d78b7fff997d1c, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=a39f8675dcfa4988af3a402b32984b84) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:38.065581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c6213-06ce-41f5-a642-6ba7b46e9375 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:38.067302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c6213-06ce-41f5-a642-6ba7b46e9375 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:38.075112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 621/1246] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:37 2022] GET /identity/v3/role_assignments?user.id=a39f8675dcfa4988af3a402b32984b84&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=c49d2fa980524a9f95d78b7fff997d1c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:40.297716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb471-00d6-4415-8b87-5774bd4a30cb None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:40.298322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb471-00d6-4415-8b87-5774bd4a30cb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:40.298744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb471-00d6-4415-8b87-5774bd4a30cb None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:40.300446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 626/1247] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:40.310474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab1b6-4a38-43d2-8454-995df61699e1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:40.310474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab1b6-4a38-43d2-8454-995df61699e1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:40.311442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab1b6-4a38-43d2-8454-995df61699e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:40.321964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:40.343290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-342ab1b6-4a38-43d2-8454-995df61699e1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:40.345564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-342ab1b6-4a38-43d2-8454-995df61699e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:40.447389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 622/1248] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:40.748769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb471-00d6-4415-8b87-5774bd4a30cb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:40.775653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:40.893417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb471-00d6-4415-8b87-5774bd4a30cb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:40.941920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6df43f-e5fd-4af8-b5a3-8c2f5370694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:40.942767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6df43f-e5fd-4af8-b5a3-8c2f5370694d None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:40.943168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6df43f-e5fd-4af8-b5a3-8c2f5370694d None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:40.944220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6df43f-e5fd-4af8-b5a3-8c2f5370694d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:40.959769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6df43f-e5fd-4af8-b5a3-8c2f5370694d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-ef6df43f-e5fd-4af8-b5a3-8c2f5370694d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:40.967226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:40.968502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:40.969640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 627/1249] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:40.976620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ab1b6-4a38-43d2-8454-995df61699e1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:41.122578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ab1b6-4a38-43d2-8454-995df61699e1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:41.168046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:41.175208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:41.175853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:41.176136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:41.176409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:41.177139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:41.177458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:41.179699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:41.191555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 623/1250] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:41.198366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6df43f-e5fd-4af8-b5a3-8c2f5370694d None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:41.323353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:41.358213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6df43f-e5fd-4af8-b5a3-8c2f5370694d None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:41.409721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:41.410130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:41.410412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:41.410682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:41.424480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:41.426645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:41.449383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:41.451711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:41.453891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:41.453891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:41.453891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:41.453891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 628/1251] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:41.458910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:41.594966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf883e-cf0d-475e-beec-8e244d86a12d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:41.649492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:41.650195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:41.650481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:41.650757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:41.651523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:41.652061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:41.653605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:41.676283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 624/1252] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:41.684402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:41.842671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33957382-7bbc-4fb1-95a9-f725671247e3 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:41.879048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:41.892196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:41.892901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:41.893228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:41.893495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:41.903778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:41.905429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:07:41.921985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:41.923541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:41.925311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 629/1253] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:07:41.934852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.013587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:42.076891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e077192-578e-4907-97c4-1b439d6af256 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:42.120074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:42.120865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:42.121158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:42.121424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:42.122361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:42.122701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:42.124379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:42.153587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 625/1254] 10.222.0.95 () {56 vars in 1143 bytes} [Wed Aug 31 19:07:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:42.159573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.341389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aca93a-dbdf-4bb0-9e16-4c9f79f02160 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:42.395495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2ced7-13bb-45aa-9591-de30cb6fc6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:42.396124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2ced7-13bb-45aa-9591-de30cb6fc6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:42.397031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2ced7-13bb-45aa-9591-de30cb6fc6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:42.397305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2ced7-13bb-45aa-9591-de30cb6fc6e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:42.467126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:42.486450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2ced7-13bb-45aa-9591-de30cb6fc6e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:42.487959 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2ced7-13bb-45aa-9591-de30cb6fc6e0 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:42.630602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 630/1255] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 19:07:42.637682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.782325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032e0b7-27f4-428e-a601-d7fd936b944c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:42.798002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:42.821742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:42.822555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:42.822842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:42.823123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:42.836117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:42.837630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:42.857161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:42.859364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:42.861519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:42.861519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:42.861519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:42.861519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 626/1256] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:07:42.866969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2ced7-13bb-45aa-9591-de30cb6fc6e0 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:42.984884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:43.003013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2ced7-13bb-45aa-9591-de30cb6fc6e0 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:43.055452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:43.056102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:43.056463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:43.056776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:43.057450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:43.057757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:43.059505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:43.076471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 631/1257] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:43.085615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:43.263144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525918d-5459-4519-bbba-7a43bc42471d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:43.307022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:43.307789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:43.308071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:43.308363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:43.320441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:43.323054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:07:43.345692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:43.348193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:43.349156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 627/1258] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:07:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:43.355703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:43.496328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183be28-c0a4-4b31-85ca-18a620d73069 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:43.543447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404550a-32ec-424e-ace3-21a731c14aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:43.544113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404550a-32ec-424e-ace3-21a731c14aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:43.544447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404550a-32ec-424e-ace3-21a731c14aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:43.544828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404550a-32ec-424e-ace3-21a731c14aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:43.545648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404550a-32ec-424e-ace3-21a731c14aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:43.546048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404550a-32ec-424e-ace3-21a731c14aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:43.547897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404550a-32ec-424e-ace3-21a731c14aa7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:43.562876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 632/1259] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:07:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:43.573298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:43.610444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:43.729372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3612d0-f637-4442-b43b-6dae01212050 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:43.781294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636a6fc-f8b7-4a7a-b621-e001803cab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:43.782433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a6fc-f8b7-4a7a-b621-e001803cab72 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:43.782996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a6fc-f8b7-4a7a-b621-e001803cab72 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:43.783556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a6fc-f8b7-4a7a-b621-e001803cab72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:43.785009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636a6fc-f8b7-4a7a-b621-e001803cab72 None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=a39f8675dcfa4988af3a402b32984b84) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:43.790108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636a6fc-f8b7-4a7a-b621-e001803cab72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:43.790108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636a6fc-f8b7-4a7a-b621-e001803cab72 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:43.799709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 628/1260] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:43 2022] GET /identity/v3/role_assignments?user.id=a39f8675dcfa4988af3a402b32984b84&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:45.824734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ab257-9c1d-4f5a-9964-4a84378c718f None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:45.825564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ab257-9c1d-4f5a-9964-4a84378c718f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:45.825760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ab257-9c1d-4f5a-9964-4a84378c718f None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:45.827115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 633/1261] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:45.833043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cedc42-a842-4d3a-9d22-a3a40e2a9bfc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:45.835988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cedc42-a842-4d3a-9d22-a3a40e2a9bfc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:45.835988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cedc42-a842-4d3a-9d22-a3a40e2a9bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:45.854805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-11cedc42-a842-4d3a-9d22-a3a40e2a9bfc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:45.855995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-11cedc42-a842-4d3a-9d22-a3a40e2a9bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:45.936207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 629/1262] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:07:46.225639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ab257-9c1d-4f5a-9964-4a84378c718f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:46.237104 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:46.339130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ab257-9c1d-4f5a-9964-4a84378c718f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:46.375340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009a8ad-7b60-408f-8994-8220d28d1744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:46.375870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009a8ad-7b60-408f-8994-8220d28d1744 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:46.376234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009a8ad-7b60-408f-8994-8220d28d1744 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:46.376533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009a8ad-7b60-408f-8994-8220d28d1744 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:46.388085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009a8ad-7b60-408f-8994-8220d28d1744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-9009a8ad-7b60-408f-8994-8220d28d1744 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:46.393666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:46.394589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:46.395428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 634/1263] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:46.401708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cedc42-a842-4d3a-9d22-a3a40e2a9bfc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:46.426242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:46.506282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cedc42-a842-4d3a-9d22-a3a40e2a9bfc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:46.540171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:46.540800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:46.541081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:46.541353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:46.542009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:46.542309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:46.543677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:46.552490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 630/1264] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:46.560022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009a8ad-7b60-408f-8994-8220d28d1744 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:46.633450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:46.676204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009a8ad-7b60-408f-8994-8220d28d1744 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:46.715023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:46.715648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:46.715926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:46.716197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:46.727156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:46.728818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:46.745171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:46.746931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:46.748737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:46.748737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:46.748737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:46.748737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 635/1265] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:46.752671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:46.852830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:46.875024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20fa4f-0fc3-4f97-a67d-a834fce83435 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:46.911464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:46.912628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:46.913589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:46.913868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:46.914521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:46.914828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:46.916196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:46.931889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 631/1266] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:46.940076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:47.061767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26ae11-ffeb-4b75-9ed4-1b9bb15a5881 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:47.083574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:47.105895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:47.106753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:47.107169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:47.107529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:47.119513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:47.122027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:07:47.139239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:47.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:47.141396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 636/1267] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:47.147392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:47.262990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad61fd-0e8a-4c20-ba34-c7171804b303 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:47.306941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:47.307545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:47.307827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:47.308103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:47.308861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:47.309164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:47.310595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:47.324528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:47.334674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 632/1268] 10.222.0.95 () {56 vars in 1143 bytes} [Wed Aug 31 19:07:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:47.340516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:47.441057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e94c7-1ee0-4974-9d65-e8bf2c5ea019 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:47.479115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:47.479743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:47.480022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:47.480294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:47.492445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:47.493953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:47.508963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:47.510146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:47.511305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:47.511305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:47.511305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:47.511305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 637/1269] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:47.516283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:47.624288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4243b98-dea5-47e8-af92-3fad211eaea0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:47.669237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:47.669832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:47.670108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:47.670371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:47.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:47.671564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:47.672946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:47.684713 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 633/1270] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:47.691303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:47.733852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:47.817861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16202f-9711-4ddb-adb0-4f50c87d3898 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:47.855042 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:47.856018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:47.856461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:47.856936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:47.866108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:47.867701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:07:47.879328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:47.881698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:47.883482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 638/1271] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:47.890115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:47.962920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:48.009317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d4be-c122-41f0-a1f4-79022cb64230 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:48.043641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309b02f-6712-4d31-9852-d7062fd8d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:48.044239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309b02f-6712-4d31-9852-d7062fd8d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:48.044520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309b02f-6712-4d31-9852-d7062fd8d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:48.044811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309b02f-6712-4d31-9852-d7062fd8d9a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:48.045487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309b02f-6712-4d31-9852-d7062fd8d9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:48.045791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309b02f-6712-4d31-9852-d7062fd8d9a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:48.047470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309b02f-6712-4d31-9852-d7062fd8d9a8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:48.060965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 634/1272] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:07:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:48.067245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:48.180855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:48.191897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca2ddc-bc44-460e-8f08-fe7bd8b2d209 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:48.227447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e9866-d6e5-46c7-9371-cba27c3c2ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:48.228107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e9866-d6e5-46c7-9371-cba27c3c2ae5 None admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:48.228391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e9866-d6e5-46c7-9371-cba27c3c2ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:48.228688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e9866-d6e5-46c7-9371-cba27c3c2ae5 None admin] PATH_INFO: `/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/a39f8675dcfa4988af3a402b32984b84/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:48.230638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e9866-d6e5-46c7-9371-cba27c3c2ae5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5680599cf08e4aecbd6b34fd082197c8, user_id=a39f8675dcfa4988af3a402b32984b84, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:48.232133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e9866-d6e5-46c7-9371-cba27c3c2ae5 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:48.246905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 639/1273] 10.222.0.95 () {58 vars in 1439 bytes} [Wed Aug 31 19:07:48 2022] PUT /identity/v3/projects/5680599cf08e4aecbd6b34fd082197c8/users/a39f8675dcfa4988af3a402b32984b84/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:07:50.263715 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1054a-6cb5-4e18-91b9-4ed54a6c941d None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:50.264241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1054a-6cb5-4e18-91b9-4ed54a6c941d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:50.264241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1054a-6cb5-4e18-91b9-4ed54a6c941d None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:50.265611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 635/1274] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:50.271225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53ad8-fab5-4c2b-8fd8-4b1112110f4f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:50.272242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53ad8-fab5-4c2b-8fd8-4b1112110f4f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:50.272322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53ad8-fab5-4c2b-8fd8-4b1112110f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:50.290633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e6b53ad8-fab5-4c2b-8fd8-4b1112110f4f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:50.292009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e6b53ad8-fab5-4c2b-8fd8-4b1112110f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:50.360289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 640/1275] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:50.636914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1054a-6cb5-4e18-91b9-4ed54a6c941d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:50.750645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1054a-6cb5-4e18-91b9-4ed54a6c941d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:50.767921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:50.785428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a5b6b-8977-4ec8-99cd-434a06c53e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:50.786058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a5b6b-8977-4ec8-99cd-434a06c53e3d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:50.786336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a5b6b-8977-4ec8-99cd-434a06c53e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:50.786604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a5b6b-8977-4ec8-99cd-434a06c53e3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:50.797527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a5b6b-8977-4ec8-99cd-434a06c53e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-182a5b6b-8977-4ec8-99cd-434a06c53e3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:07:50.802322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:50.804050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:50.805123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 636/1276] 10.222.0.95 () {56 vars in 1096 bytes} [Wed Aug 31 19:07:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:50.810060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b53ad8-fab5-4c2b-8fd8-4b1112110f4f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:50.899867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:50.919483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b53ad8-fab5-4c2b-8fd8-4b1112110f4f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:50.963696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:50.964504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:50.964886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:50.965280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:50.966207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:50.966665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:50.968652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:50.981356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 641/1277] 10.222.0.95 () {56 vars in 1099 bytes} [Wed Aug 31 19:07:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:50.987162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a5b6b-8977-4ec8-99cd-434a06c53e3d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:51.109201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a5b6b-8977-4ec8-99cd-434a06c53e3d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:51.147769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:51.148404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:51.148718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:51.148988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:51.160933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:51.162388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:51.178181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:51.179364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:51.180275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:51.180275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:51.180275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:51.180275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 637/1278] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:51.185512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:51.286160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daccc52-b64b-4c70-b3f4-80e40e9b798c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:51.320923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:51.321530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:51.321815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:51.322084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:51.322729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:51.323030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:51.324414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:51.336643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 642/1279] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:51.342615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:51.375562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:51.452494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0ac5-823a-4375-b2b3-471985a4bdbf None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:51.485604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:51.486216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:51.486492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:51.486761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:51.494665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:51.496086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:07:51.507569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:51.508881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:51.510092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 638/1280] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:51.514814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:51.518598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:51.619899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615795bc-e344-4895-b3e4-7242859157c4 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:51.658122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:51.658721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:51.658997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:51.659258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:51.659979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:51.660275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:51.662276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:51.680100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 643/1281] 10.222.0.95 () {56 vars in 1143 bytes} [Wed Aug 31 19:07:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:51.686102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:51.785487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:51.797278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78ab9d-2e66-44da-945f-698a06d6520b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:51.831867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2ee91-3cb9-4e8b-88f6-269eef374774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:51.832474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2ee91-3cb9-4e8b-88f6-269eef374774 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:51.833000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2ee91-3cb9-4e8b-88f6-269eef374774 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:51.833450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2ee91-3cb9-4e8b-88f6-269eef374774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:51.902754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2ee91-3cb9-4e8b-88f6-269eef374774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:51.904204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2ee91-3cb9-4e8b-88f6-269eef374774 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:51.996968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:52.009461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 639/1282] 10.222.0.95 () {58 vars in 1282 bytes} [Wed Aug 31 19:07:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:07:52.015716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:52.079802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:52.124262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5173a8d-1ea9-4893-922f-2064ef3a0f2a None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:52.159769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:52.160383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:52.160698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:52.161015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:52.172304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:52.173809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:07:52.188166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:52.189269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:07:52.190192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:07:52.190192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:07:52.190192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:52.190192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 644/1283] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:52.195123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2ee91-3cb9-4e8b-88f6-269eef374774 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:52.299939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2ee91-3cb9-4e8b-88f6-269eef374774 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:52.335290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:52.335953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:52.336233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:52.336504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:52.337178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:52.337479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:52.338849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:52.350776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 640/1284] 10.222.0.95 () {56 vars in 1109 bytes} [Wed Aug 31 19:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:52.356603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:52.479426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db200605-5fa2-493c-be3e-6c633c53c47c None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:52.493727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:52.522780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:52.523400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:52.523675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:52.523943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:52.530672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:52.532119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:07:52.543729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:52.544867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: ERROR keystone.server.flask.application Aug 31 19:07:52.545806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 645/1285] 10.222.0.95 () {56 vars in 1111 bytes} [Wed Aug 31 19:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:07:52.551048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:52.607426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:52.663048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92df40-3ee1-4e8f-be85-33faf82e0382 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:52.700404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca79c51-f73a-481d-8e37-803bf7eb8169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:52.701039 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79c51-f73a-481d-8e37-803bf7eb8169 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:52.701320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79c51-f73a-481d-8e37-803bf7eb8169 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:52.701588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca79c51-f73a-481d-8e37-803bf7eb8169 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:52.702272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca79c51-f73a-481d-8e37-803bf7eb8169 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:52.702574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca79c51-f73a-481d-8e37-803bf7eb8169 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:52.703986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca79c51-f73a-481d-8e37-803bf7eb8169 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:52.716290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 641/1286] 10.222.0.95 () {56 vars in 1148 bytes} [Wed Aug 31 19:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:52.722430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:52.831220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35daa0c0-b9c0-4218-ba70-0fefee087ea7 None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:52.867010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f08c3-09fc-4f43-b1c6-6a29892ee90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:52.867529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f08c3-09fc-4f43-b1c6-6a29892ee90b None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:52.867802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f08c3-09fc-4f43-b1c6-6a29892ee90b None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:52.868066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f08c3-09fc-4f43-b1c6-6a29892ee90b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:52.868815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f08c3-09fc-4f43-b1c6-6a29892ee90b None admin] RBAC: Adding query filter params (role.id=ece89e7759bc4d7891cb77a70f05974f, scope.project.id=5680599cf08e4aecbd6b34fd082197c8, user.id=a39f8675dcfa4988af3a402b32984b84) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:52.869121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f08c3-09fc-4f43-b1c6-6a29892ee90b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:52.870545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f08c3-09fc-4f43-b1c6-6a29892ee90b None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:52.876383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 646/1287] 10.222.0.95 () {56 vars in 1374 bytes} [Wed Aug 31 19:07:52 2022] GET /identity/v3/role_assignments?user.id=a39f8675dcfa4988af3a402b32984b84&scope.project.id=5680599cf08e4aecbd6b34fd082197c8&role.id=ece89e7759bc4d7891cb77a70f05974f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:54.931815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b14a0-97d0-48f8-983c-2946756b1879 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:54.932436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b14a0-97d0-48f8-983c-2946756b1879 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:54.932436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b14a0-97d0-48f8-983c-2946756b1879 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:54.934067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 642/1288] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:54.940100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac5a7c-a545-43d0-af28-322e6f462741 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:54.940599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac5a7c-a545-43d0-af28-322e6f462741 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:54.940878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac5a7c-a545-43d0-af28-322e6f462741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:54.960709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-5fac5a7c-a545-43d0-af28-322e6f462741 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:54.961937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5fac5a7c-a545-43d0-af28-322e6f462741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:54.993832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:55.037618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 647/1289] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:55.320476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b14a0-97d0-48f8-983c-2946756b1879 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:55.433513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b14a0-97d0-48f8-983c-2946756b1879 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:55.446445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:55.472805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:55.473480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:55.473762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:55.474030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:55.475191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:55.476606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:07:55.486422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:07:55.487833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:07:55.488759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 643/1290] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:07:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:07:55.494487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac5a7c-a545-43d0-af28-322e6f462741 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:55.599859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac5a7c-a545-43d0-af28-322e6f462741 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:55.642268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf4d3e-251a-4294-9b5c-b7192bd3d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:55.642923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf4d3e-251a-4294-9b5c-b7192bd3d57f None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:55.643270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf4d3e-251a-4294-9b5c-b7192bd3d57f None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:55.643552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf4d3e-251a-4294-9b5c-b7192bd3d57f None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:55.644295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf4d3e-251a-4294-9b5c-b7192bd3d57f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:55.644659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf4d3e-251a-4294-9b5c-b7192bd3d57f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:55.646339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf4d3e-251a-4294-9b5c-b7192bd3d57f None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:55.657879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 648/1291] 10.222.0.95 () {56 vars in 1116 bytes} [Wed Aug 31 19:07:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:07:55.659449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:55.777416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea08f1-d503-4491-b57e-48031ed7e6c7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:55.815465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282ec06-171e-474c-8cbe-cb1e92349352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:55.816126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282ec06-171e-474c-8cbe-cb1e92349352 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:55.816434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282ec06-171e-474c-8cbe-cb1e92349352 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:55.816762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282ec06-171e-474c-8cbe-cb1e92349352 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:55.817438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282ec06-171e-474c-8cbe-cb1e92349352 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:07:55.817775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282ec06-171e-474c-8cbe-cb1e92349352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:55.819608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282ec06-171e-474c-8cbe-cb1e92349352 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:55.824786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 644/1292] 10.222.0.95 () {56 vars in 1116 bytes} [Wed Aug 31 19:07:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:07:57.869431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcf16f-0305-486a-9057-cf8f4d13df46 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:57.870015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcf16f-0305-486a-9057-cf8f4d13df46 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:57.870015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcf16f-0305-486a-9057-cf8f4d13df46 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:57.871326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 649/1293] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:07:57.876848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab149f4-ae31-45d8-b495-ebc2cd106928 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:57.877313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab149f4-ae31-45d8-b495-ebc2cd106928 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:57.877405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab149f4-ae31-45d8-b495-ebc2cd106928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:57.896736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6ab149f4-ae31-45d8-b495-ebc2cd106928 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:57.897952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6ab149f4-ae31-45d8-b495-ebc2cd106928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:57.937397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:57.980140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 645/1294] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:07:58.258648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcf16f-0305-486a-9057-cf8f4d13df46 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:58.293518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:58.378239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcf16f-0305-486a-9057-cf8f4d13df46 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:58.429052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff4ff4-2fd4-4844-ba75-93d897059739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:58.429683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff4ff4-2fd4-4844-ba75-93d897059739 None admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:58.429964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff4ff4-2fd4-4844-ba75-93d897059739 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:58.430235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff4ff4-2fd4-4844-ba75-93d897059739 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:58.430918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff4ff4-2fd4-4844-ba75-93d897059739 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:58.432350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff4ff4-2fd4-4844-ba75-93d897059739 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:58.448630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 650/1295] 10.222.0.95 () {60 vars in 1141 bytes} [Wed Aug 31 19:07:58 2022] POST /identity/v3/services => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:08:00.462741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d470-aa3c-497a-afbd-6d4f81376099 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:00.463278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d470-aa3c-497a-afbd-6d4f81376099 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:00.463278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d470-aa3c-497a-afbd-6d4f81376099 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:00.464529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 646/1296] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:08:00.470068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380109a-17d4-4349-86da-8f373c466ee1 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:00.470539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380109a-17d4-4349-86da-8f373c466ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:00.471017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380109a-17d4-4349-86da-8f373c466ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:00.490848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b380109a-17d4-4349-86da-8f373c466ee1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:00.492059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b380109a-17d4-4349-86da-8f373c466ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:00.568206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 651/1297] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:08:00 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:08:00.844468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d470-aa3c-497a-afbd-6d4f81376099 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:00.946716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d470-aa3c-497a-afbd-6d4f81376099 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:00.981798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:00.982416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:00.982691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:00.982956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:00.983529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:00.984914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:08:00.994008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:08:00.995190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:08:00.996173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 647/1298] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:08:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:08:01.001198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380109a-17d4-4349-86da-8f373c466ee1 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:01.019776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:01.100265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380109a-17d4-4349-86da-8f373c466ee1 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:01.136280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:01.136917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:01.137194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:01.137462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:01.138092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:01.138383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:01.139772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:01.144923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 652/1299] 10.222.0.95 () {56 vars in 1116 bytes} [Wed Aug 31 19:08:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:08:01.155746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:01.189140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:01.261249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc79df0-d6c5-47fd-b963-4438d04c4c57 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:01.304742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4100a98-e832-40c6-aa96-ce1e7cfac4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:01.305344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100a98-e832-40c6-aa96-ce1e7cfac4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:01.305622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100a98-e832-40c6-aa96-ce1e7cfac4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:01.305890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100a98-e832-40c6-aa96-ce1e7cfac4f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:01.306576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4100a98-e832-40c6-aa96-ce1e7cfac4f5 None admin] RBAC: Adding query filter params (interface=public, service_id=7f6c3fe8b915421796f0da926a4b1a6a, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:01.306874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4100a98-e832-40c6-aa96-ce1e7cfac4f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:01.308234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4100a98-e832-40c6-aa96-ce1e7cfac4f5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:01.313185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 648/1300] 10.222.0.95 () {56 vars in 1251 bytes} [Wed Aug 31 19:08:01 2022] GET /identity/v3/endpoints?service_id=7f6c3fe8b915421796f0da926a4b1a6a&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:08:01.319921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:01.401358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:01.420015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d791f-cd07-4e63-88ec-2e5feef744bc None admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:01.455296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98336592-223c-4d0b-80d8-fc57dafc8837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:01.455893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336592-223c-4d0b-80d8-fc57dafc8837 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:01.456170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336592-223c-4d0b-80d8-fc57dafc8837 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:01.456433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336592-223c-4d0b-80d8-fc57dafc8837 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:01.457063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98336592-223c-4d0b-80d8-fc57dafc8837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:01.458516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98336592-223c-4d0b-80d8-fc57dafc8837 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:01.465441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 653/1301] 10.222.0.95 () {56 vars in 1087 bytes} [Wed Aug 31 19:08:01 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:08:03.375863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2146-3b5e-476b-9c4d-4f9bc6657f1f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:03.376402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2146-3b5e-476b-9c4d-4f9bc6657f1f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:03.376402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d2146-3b5e-476b-9c4d-4f9bc6657f1f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:03.377717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 649/1302] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:08:03.383610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e3ddf-9407-45b1-b6c2-f53c017adfb7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:03.384040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e3ddf-9407-45b1-b6c2-f53c017adfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:03.384225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e3ddf-9407-45b1-b6c2-f53c017adfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:03.402259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-309e3ddf-9407-45b1-b6c2-f53c017adfb7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:03.403438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-309e3ddf-9407-45b1-b6c2-f53c017adfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:03.471774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 654/1303] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:08:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:08:03.745524 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d2146-3b5e-476b-9c4d-4f9bc6657f1f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:03.843367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:03.851299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d2146-3b5e-476b-9c4d-4f9bc6657f1f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:03.887225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:03.888090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:03.888469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:03.888903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:03.889718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:03.891729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:08:03.901391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:08:03.902533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Aug 31 19:08:03.903408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 650/1304] 10.222.0.95 () {56 vars in 1117 bytes} [Wed Aug 31 19:08:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:08:03.908793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e3ddf-9407-45b1-b6c2-f53c017adfb7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:03.988844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:04.013422 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e3ddf-9407-45b1-b6c2-f53c017adfb7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:04.050434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce185-e741-4601-a1be-a4660d1d9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:04.051057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce185-e741-4601-a1be-a4660d1d9a54 None admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:04.051380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce185-e741-4601-a1be-a4660d1d9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:04.051606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce185-e741-4601-a1be-a4660d1d9a54 None admin] PATH_INFO: `/v3/services` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:04.052349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce185-e741-4601-a1be-a4660d1d9a54 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:04.053289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce185-e741-4601-a1be-a4660d1d9a54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:04.054693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce185-e741-4601-a1be-a4660d1d9a54 None admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:04.060219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 655/1305] 10.222.0.95 () {56 vars in 1116 bytes} [Wed Aug 31 19:08:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:08:04.067139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:04.174052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57012ca-ea37-4f11-b389-0e564e935888 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:04.208889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae83a6b-bf96-4f10-a422-316058a9a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:04.209515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae83a6b-bf96-4f10-a422-316058a9a20a None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:04.209789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae83a6b-bf96-4f10-a422-316058a9a20a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:04.210056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae83a6b-bf96-4f10-a422-316058a9a20a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:04.210730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae83a6b-bf96-4f10-a422-316058a9a20a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:04.212090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae83a6b-bf96-4f10-a422-316058a9a20a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:04.232614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 651/1306] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:08:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:08:14.057658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=30 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:14.228147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:11.649465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd9e4-183f-4c0a-936c-308a54419817 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:11.650020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd9e4-183f-4c0a-936c-308a54419817 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:11.650020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd9e4-183f-4c0a-936c-308a54419817 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:11.651453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 656/1307] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:11.656677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0e4c2-6679-4dd5-9a0a-dc20517fae5c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:11.657181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0e4c2-6679-4dd5-9a0a-dc20517fae5c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:11.657265 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0e4c2-6679-4dd5-9a0a-dc20517fae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:11.772252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c9c0e4c2-6679-4dd5-9a0a-dc20517fae5c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:11.773512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c9c0e4c2-6679-4dd5-9a0a-dc20517fae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:11.960606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 652/1308] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:12.257919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ae69-0136-4916-adc9-bbf2ae87eddb None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:12.258609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ae69-0136-4916-adc9-bbf2ae87eddb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:12.259119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ae69-0136-4916-adc9-bbf2ae87eddb None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:12.260980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 657/1309] 10.222.0.95 () {54 vars in 965 bytes} [Wed Aug 31 19:09:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:12.266771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711d6bf-c2e5-4505-b9c2-40bd7e97d01a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:12.267106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711d6bf-c2e5-4505-b9c2-40bd7e97d01a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:12.267338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711d6bf-c2e5-4505-b9c2-40bd7e97d01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:12.359449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e711d6bf-c2e5-4505-b9c2-40bd7e97d01a None None] MFA Rules not processed for user `5f1d5c29df30465ead9834b63c3e93bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:12.363823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e711d6bf-c2e5-4505-b9c2-40bd7e97d01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:12.397158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:12.484411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 653/1310] 10.222.0.95 () {58 vars in 1062 bytes} [Wed Aug 31 19:09:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:12.495014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ae69-0136-4916-adc9-bbf2ae87eddb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:12.496918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3e44ae69-0136-4916-adc9-bbf2ae87eddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:12.637304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ae69-0136-4916-adc9-bbf2ae87eddb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:12.742028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659466-dea1-485d-b389-cb5c4fb50a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:12.743374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659466-dea1-485d-b389-cb5c4fb50a06 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:12.743812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659466-dea1-485d-b389-cb5c4fb50a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:12.744254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659466-dea1-485d-b389-cb5c4fb50a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:12.746172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1b659466-dea1-485d-b389-cb5c4fb50a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:12.814101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:12.879238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b659466-dea1-485d-b389-cb5c4fb50a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:12.880742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b659466-dea1-485d-b389-cb5c4fb50a06 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:12.902334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 658/1311] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:15.177458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc6b8c-df7a-43d2-bef8-8d0794c53351 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:15.177993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc6b8c-df7a-43d2-bef8-8d0794c53351 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:15.177993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc6b8c-df7a-43d2-bef8-8d0794c53351 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:15.179351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 654/1312] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:15.184876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa36dc6-3d81-4b0e-b603-5a274609a11f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:15.185334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa36dc6-3d81-4b0e-b603-5a274609a11f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:15.185432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa36dc6-3d81-4b0e-b603-5a274609a11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:15.262063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-caa36dc6-3d81-4b0e-b603-5a274609a11f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:15.263315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-caa36dc6-3d81-4b0e-b603-5a274609a11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:15.373811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 659/1313] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:09:15.666815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc6b8c-df7a-43d2-bef8-8d0794c53351 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:15.668422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6afc6b8c-df7a-43d2-bef8-8d0794c53351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:15.779453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc6b8c-df7a-43d2-bef8-8d0794c53351 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:15.852877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:15.855506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34afe6-c0d4-42f1-90d0-b694f76f2e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:15.856303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34afe6-c0d4-42f1-90d0-b694f76f2e40 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:15.856801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34afe6-c0d4-42f1-90d0-b694f76f2e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:15.857267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34afe6-c0d4-42f1-90d0-b694f76f2e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:15.858921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1b34afe6-c0d4-42f1-90d0-b694f76f2e40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:15.970638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34afe6-c0d4-42f1-90d0-b694f76f2e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:15.972086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34afe6-c0d4-42f1-90d0-b694f76f2e40 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:15.990767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 655/1314] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:18.254364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036bd09-f245-44ab-9fd0-1ff89fc33554 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:18.254912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036bd09-f245-44ab-9fd0-1ff89fc33554 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:18.254912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036bd09-f245-44ab-9fd0-1ff89fc33554 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:18.256974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 660/1315] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:18.262269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bcf24-9d5e-4309-9163-a5cda1f44e7e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:18.262686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bcf24-9d5e-4309-9163-a5cda1f44e7e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:18.262821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bcf24-9d5e-4309-9163-a5cda1f44e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:18.329167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-703bcf24-9d5e-4309-9163-a5cda1f44e7e None None] MFA Rules not processed for user `99d2701b53de47caa5ad2c67fb1f0e64`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:18.330382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-703bcf24-9d5e-4309-9163-a5cda1f44e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:18.383511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:18.428959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 656/1316] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:18.720437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036bd09-f245-44ab-9fd0-1ff89fc33554 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:18.722028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e036bd09-f245-44ab-9fd0-1ff89fc33554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:18.732931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:18.826809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036bd09-f245-44ab-9fd0-1ff89fc33554 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:18.907518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5a004-3d49-48ac-93d4-2400c1314534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:18.908131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a004-3d49-48ac-93d4-2400c1314534 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:18.908394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a004-3d49-48ac-93d4-2400c1314534 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:18.908692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a004-3d49-48ac-93d4-2400c1314534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:18.910857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2ff5a004-3d49-48ac-93d4-2400c1314534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:18.973314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:19.021315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5a004-3d49-48ac-93d4-2400c1314534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:19.022768 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5a004-3d49-48ac-93d4-2400c1314534 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:19.040581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 661/1317] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:21.511672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ef67-049b-4b91-a092-5532d90d2530 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:21.512214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ef67-049b-4b91-a092-5532d90d2530 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:21.512214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ef67-049b-4b91-a092-5532d90d2530 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:21.513599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 657/1318] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:21.518581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31109e-707d-4ff2-8b9f-2a5e8ecafdfd None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:21.519010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31109e-707d-4ff2-8b9f-2a5e8ecafdfd None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:21.519215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31109e-707d-4ff2-8b9f-2a5e8ecafdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:21.583835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-7f31109e-707d-4ff2-8b9f-2a5e8ecafdfd None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:21.585067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7f31109e-707d-4ff2-8b9f-2a5e8ecafdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:21.683755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 662/1319] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:21.969435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92ef67-049b-4b91-a092-5532d90d2530 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:21.970985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ce92ef67-049b-4b91-a092-5532d90d2530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:22.089316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92ef67-049b-4b91-a092-5532d90d2530 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:22.170379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a40b0b-40c0-4480-8fb1-1837ad8dafb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:22.170984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a40b0b-40c0-4480-8fb1-1837ad8dafb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:22.171262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a40b0b-40c0-4480-8fb1-1837ad8dafb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:22.171531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a40b0b-40c0-4480-8fb1-1837ad8dafb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:22.173138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-36a40b0b-40c0-4480-8fb1-1837ad8dafb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:22.181235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:22.286416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a40b0b-40c0-4480-8fb1-1837ad8dafb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:22.287763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a40b0b-40c0-4480-8fb1-1837ad8dafb1 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:22.306529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 658/1320] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:24.959920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abf0b0-2f69-4a41-85e4-02b7183e29cf None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:24.960471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abf0b0-2f69-4a41-85e4-02b7183e29cf None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:24.960471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abf0b0-2f69-4a41-85e4-02b7183e29cf None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:24.961829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 663/1321] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:24.967078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4ae5-ea1f-469e-ac65-15bd6c5f7d18 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:24.967528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4ae5-ea1f-469e-ac65-15bd6c5f7d18 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:24.967775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4ae5-ea1f-469e-ac65-15bd6c5f7d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:25.037668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-630c4ae5-ea1f-469e-ac65-15bd6c5f7d18 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:25.038892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-630c4ae5-ea1f-469e-ac65-15bd6c5f7d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:25.099024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:25.151471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 659/1322] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:25.447177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abf0b0-2f69-4a41-85e4-02b7183e29cf None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:25.448744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-25abf0b0-2f69-4a41-85e4-02b7183e29cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:25.468894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:25.559628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abf0b0-2f69-4a41-85e4-02b7183e29cf None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:25.634320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6183c0a-2068-4821-981a-724a2f4a355b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:25.634944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183c0a-2068-4821-981a-724a2f4a355b service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:25.635258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183c0a-2068-4821-981a-724a2f4a355b service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:25.635494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6183c0a-2068-4821-981a-724a2f4a355b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:25.637195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a6183c0a-2068-4821-981a-724a2f4a355b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:25.713131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:25.758276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6183c0a-2068-4821-981a-724a2f4a355b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:25.759744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6183c0a-2068-4821-981a-724a2f4a355b service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:25.778497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 664/1323] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:27.771113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56ec36-2b37-4994-8946-9b3cda2e09a5 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:27.771674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56ec36-2b37-4994-8946-9b3cda2e09a5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:27.771674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56ec36-2b37-4994-8946-9b3cda2e09a5 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:27.772932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 660/1324] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:27.778583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4df4-37c1-4e02-a1b0-969ab63e3e26 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:27.779135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4df4-37c1-4e02-a1b0-969ab63e3e26 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:27.779135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4df4-37c1-4e02-a1b0-969ab63e3e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:27.850291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-985e4df4-37c1-4e02-a1b0-969ab63e3e26 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:27.851499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-985e4df4-37c1-4e02-a1b0-969ab63e3e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:27.950932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 665/1325] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:28.235104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56ec36-2b37-4994-8946-9b3cda2e09a5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:28.236728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fb56ec36-2b37-4994-8946-9b3cda2e09a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:28.358927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56ec36-2b37-4994-8946-9b3cda2e09a5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:28.474457 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f70c1-3e03-465d-9cc0-3c706ce6b091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:28.475102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f70c1-3e03-465d-9cc0-3c706ce6b091 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:28.475381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f70c1-3e03-465d-9cc0-3c706ce6b091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:28.475654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f70c1-3e03-465d-9cc0-3c706ce6b091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:28.477393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dd9f70c1-3e03-465d-9cc0-3c706ce6b091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:28.503012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:28.612066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f70c1-3e03-465d-9cc0-3c706ce6b091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:28.614151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f70c1-3e03-465d-9cc0-3c706ce6b091 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:28.633315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 661/1326] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:30.671442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1a067-e1b1-48d0-9d46-c69f069056e8 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:30.672034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1a067-e1b1-48d0-9d46-c69f069056e8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:30.672034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1a067-e1b1-48d0-9d46-c69f069056e8 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:30.673510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 666/1327] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:30.680212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee2b61-651b-4b09-ab5e-49053f2d8f50 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:30.680801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee2b61-651b-4b09-ab5e-49053f2d8f50 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:30.680801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee2b61-651b-4b09-ab5e-49053f2d8f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:30.746598 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d6ee2b61-651b-4b09-ab5e-49053f2d8f50 None None] MFA Rules not processed for user `99d2701b53de47caa5ad2c67fb1f0e64`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:30.747856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d6ee2b61-651b-4b09-ab5e-49053f2d8f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:30.812021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:30.845123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 662/1328] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:31.154325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854d15-0cf9-44e5-9d87-489fdd2085a4 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:31.154516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854d15-0cf9-44e5-9d87-489fdd2085a4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:31.154785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38854d15-0cf9-44e5-9d87-489fdd2085a4 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:31.156124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 667/1329] 10.222.0.95 () {54 vars in 965 bytes} [Wed Aug 31 19:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:31.163753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28993-d05d-460f-908b-163f0edfe39c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:31.164052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28993-d05d-460f-908b-163f0edfe39c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:31.164294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e28993-d05d-460f-908b-163f0edfe39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:31.244152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a9e28993-d05d-460f-908b-163f0edfe39c None None] MFA Rules not processed for user `5f1d5c29df30465ead9834b63c3e93bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:31.245390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a9e28993-d05d-460f-908b-163f0edfe39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:31.321054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 663/1330] 10.222.0.95 () {58 vars in 1062 bytes} [Wed Aug 31 19:09:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:31.330101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854d15-0cf9-44e5-9d87-489fdd2085a4 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:31.331742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-38854d15-0cf9-44e5-9d87-489fdd2085a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:31.367575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:31.440486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38854d15-0cf9-44e5-9d87-489fdd2085a4 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:31.515887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05417b-6af6-4ec7-ac9a-160eca036e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:31.516514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05417b-6af6-4ec7-ac9a-160eca036e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:31.516821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05417b-6af6-4ec7-ac9a-160eca036e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:31.517092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05417b-6af6-4ec7-ac9a-160eca036e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:31.519215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0e05417b-6af6-4ec7-ac9a-160eca036e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:31.598791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:31.631787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05417b-6af6-4ec7-ac9a-160eca036e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:31.633289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05417b-6af6-4ec7-ac9a-160eca036e99 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:31.651157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 668/1331] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:34.290880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0a635-da57-421b-ad02-83b6760df9a3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:34.291446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0a635-da57-421b-ad02-83b6760df9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:34.291446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0a635-da57-421b-ad02-83b6760df9a3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:34.292796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 664/1332] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:34.298145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff4522-fc39-4d7c-974f-1558fd1d715b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:34.298599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff4522-fc39-4d7c-974f-1558fd1d715b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:34.298697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff4522-fc39-4d7c-974f-1558fd1d715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:34.367640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-48ff4522-fc39-4d7c-974f-1558fd1d715b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:34.368883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-48ff4522-fc39-4d7c-974f-1558fd1d715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:34.467202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 669/1333] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:09:34.748318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0a635-da57-421b-ad02-83b6760df9a3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:34.749873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bdb0a635-da57-421b-ad02-83b6760df9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:34.820260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:34.855237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0a635-da57-421b-ad02-83b6760df9a3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:34.934375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f98d4-7cce-4e9c-a13e-d6b1e84254d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:34.934971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f98d4-7cce-4e9c-a13e-d6b1e84254d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:34.935313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f98d4-7cce-4e9c-a13e-d6b1e84254d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:34.935525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f98d4-7cce-4e9c-a13e-d6b1e84254d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:34.937166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fa4f98d4-7cce-4e9c-a13e-d6b1e84254d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:35.046044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f98d4-7cce-4e9c-a13e-d6b1e84254d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:35.047400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f98d4-7cce-4e9c-a13e-d6b1e84254d6 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:35.057814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:35.066686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 665/1334] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:37.045290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53558-2d72-48a1-9e8f-50cb89f12f72 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:37.045867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53558-2d72-48a1-9e8f-50cb89f12f72 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:37.045867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53558-2d72-48a1-9e8f-50cb89f12f72 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:37.047185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 670/1335] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:37.052538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebc5d5-208c-4b87-9b15-5cea7255e8df None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:37.052981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebc5d5-208c-4b87-9b15-5cea7255e8df None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:37.053129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebc5d5-208c-4b87-9b15-5cea7255e8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:37.120344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d7ebc5d5-208c-4b87-9b15-5cea7255e8df None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:37.121594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d7ebc5d5-208c-4b87-9b15-5cea7255e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:37.220270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 666/1336] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:37.514160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53558-2d72-48a1-9e8f-50cb89f12f72 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:37.515691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-71b53558-2d72-48a1-9e8f-50cb89f12f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:37.556391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:37.620994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53558-2d72-48a1-9e8f-50cb89f12f72 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:37.696702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca22c16-3e52-44b3-8858-ef52d951a62a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:37.697307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca22c16-3e52-44b3-8858-ef52d951a62a service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:37.697588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca22c16-3e52-44b3-8858-ef52d951a62a service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:37.697858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca22c16-3e52-44b3-8858-ef52d951a62a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:37.699477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8ca22c16-3e52-44b3-8858-ef52d951a62a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:37.784154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:37.810994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca22c16-3e52-44b3-8858-ef52d951a62a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:37.812463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca22c16-3e52-44b3-8858-ef52d951a62a service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:37.830989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 671/1337] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:40.255491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47cee5-6f73-42e4-8fd5-8472c1811d4f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:40.256089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47cee5-6f73-42e4-8fd5-8472c1811d4f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:40.256089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47cee5-6f73-42e4-8fd5-8472c1811d4f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:40.257432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 667/1338] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:40.262661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55994-01e3-4bc5-af0c-485f91938203 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:40.263102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55994-01e3-4bc5-af0c-485f91938203 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:40.263203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55994-01e3-4bc5-af0c-485f91938203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:40.329378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-44c55994-01e3-4bc5-af0c-485f91938203 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:40.330601 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-44c55994-01e3-4bc5-af0c-485f91938203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:40.429907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 672/1339] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:40.718787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47cee5-6f73-42e4-8fd5-8472c1811d4f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:40.720310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7c47cee5-6f73-42e4-8fd5-8472c1811d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:40.803555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:40.831213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47cee5-6f73-42e4-8fd5-8472c1811d4f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:40.904359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b8fdc-c63a-4b0b-878f-cabed5d60d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:40.905010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8fdc-c63a-4b0b-878f-cabed5d60d64 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:40.905298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8fdc-c63a-4b0b-878f-cabed5d60d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:40.905572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8fdc-c63a-4b0b-878f-cabed5d60d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:40.907228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b22b8fdc-c63a-4b0b-878f-cabed5d60d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:41.017231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b8fdc-c63a-4b0b-878f-cabed5d60d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:41.018974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b8fdc-c63a-4b0b-878f-cabed5d60d64 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:41.037427 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 668/1340] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:45.036287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99109-3184-424d-8e73-c1220f915b6a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:45.036849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99109-3184-424d-8e73-c1220f915b6a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:45.036849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99109-3184-424d-8e73-c1220f915b6a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:45.038349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 673/1341] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:45.043528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4ee1-eef2-45b9-8bd9-fc022eafee6c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:45.044250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4ee1-eef2-45b9-8bd9-fc022eafee6c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:45.044487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd4ee1-eef2-45b9-8bd9-fc022eafee6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:45.051052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:45.119267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-23fd4ee1-eef2-45b9-8bd9-fc022eafee6c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:45.120493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-23fd4ee1-eef2-45b9-8bd9-fc022eafee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:45.230159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 669/1342] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:45.519508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99109-3184-424d-8e73-c1220f915b6a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:45.521081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-54c99109-3184-424d-8e73-c1220f915b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:45.573590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:45.634800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99109-3184-424d-8e73-c1220f915b6a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:45.716074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b02420-56f8-4d07-916a-45c490c08540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:45.716718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b02420-56f8-4d07-916a-45c490c08540 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:45.717004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b02420-56f8-4d07-916a-45c490c08540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:45.717278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b02420-56f8-4d07-916a-45c490c08540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:45.719015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-69b02420-56f8-4d07-916a-45c490c08540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:45.821021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:45.839097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b02420-56f8-4d07-916a-45c490c08540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:45.840548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b02420-56f8-4d07-916a-45c490c08540 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:45.859712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 674/1343] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:48.424292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef737e5-2b96-42b3-aacd-089d4fd49221 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:48.424727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef737e5-2b96-42b3-aacd-089d4fd49221 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:48.424993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef737e5-2b96-42b3-aacd-089d4fd49221 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:48.426352 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 670/1344] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:48.431714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6390d8f-ca7b-476e-a339-0f4d133bb9b2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:48.432083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6390d8f-ca7b-476e-a339-0f4d133bb9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:48.432638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6390d8f-ca7b-476e-a339-0f4d133bb9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:48.504082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e6390d8f-ca7b-476e-a339-0f4d133bb9b2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:48.505345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e6390d8f-ca7b-476e-a339-0f4d133bb9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:48.605075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 675/1345] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:48.892057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef737e5-2b96-42b3-aacd-089d4fd49221 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:48.894014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1ef737e5-2b96-42b3-aacd-089d4fd49221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:48.984228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:49.004013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef737e5-2b96-42b3-aacd-089d4fd49221 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:49.083362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22c303-0a8e-415f-a892-2ea90f6b91b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:49.083965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22c303-0a8e-415f-a892-2ea90f6b91b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:49.084244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22c303-0a8e-415f-a892-2ea90f6b91b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:49.084512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22c303-0a8e-415f-a892-2ea90f6b91b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:49.086187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3d22c303-0a8e-415f-a892-2ea90f6b91b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:49.212149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22c303-0a8e-415f-a892-2ea90f6b91b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:49.213540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22c303-0a8e-415f-a892-2ea90f6b91b7 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:49.235282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 671/1346] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:53.201452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef322-cee4-45d2-80dd-8be82614cc75 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:53.201948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef322-cee4-45d2-80dd-8be82614cc75 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:53.202045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef322-cee4-45d2-80dd-8be82614cc75 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:53.203355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 676/1347] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:53.209413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84f004-6de0-4563-a191-02c069756f29 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:53.209866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84f004-6de0-4563-a191-02c069756f29 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:53.210117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84f004-6de0-4563-a191-02c069756f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:53.229492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:53.292679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1d84f004-6de0-4563-a191-02c069756f29 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:53.294181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1d84f004-6de0-4563-a191-02c069756f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:53.411386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 672/1348] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:09:53.700779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ef322-cee4-45d2-80dd-8be82614cc75 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:09:53.702889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-789ef322-cee4-45d2-80dd-8be82614cc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:53.778941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:53.832306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ef322-cee4-45d2-80dd-8be82614cc75 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:09:53.919895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb373450-0dda-4b46-ba25-b597ab5adfb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:09:53.920676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373450-0dda-4b46-ba25-b597ab5adfb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:53.921025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373450-0dda-4b46-ba25-b597ab5adfb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:53.921364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb373450-0dda-4b46-ba25-b597ab5adfb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:53.923060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cb373450-0dda-4b46-ba25-b597ab5adfb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:54.042036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:09:54.047038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb373450-0dda-4b46-ba25-b597ab5adfb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:09:54.048609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb373450-0dda-4b46-ba25-b597ab5adfb1 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:09:54.068957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 677/1349] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:09:59.168948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5ea3a-970f-48d9-8566-2915695ea3ec None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:59.169874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5ea3a-970f-48d9-8566-2915695ea3ec None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:59.170322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5ea3a-970f-48d9-8566-2915695ea3ec None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:59.173741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 673/1350] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:09:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:09:59.179363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cacea9-633a-4ea9-bbd5-1ff46ab491a5 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:09:59.179814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cacea9-633a-4ea9-bbd5-1ff46ab491a5 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:09:59.179894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cacea9-633a-4ea9-bbd5-1ff46ab491a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:09:59.296439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-82cacea9-633a-4ea9-bbd5-1ff46ab491a5 None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:09:59.306328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-82cacea9-633a-4ea9-bbd5-1ff46ab491a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:09:59.471336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 678/1351] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:09:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:03.410629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=59 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:03.670327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec91c-c270-4811-b18c-c8123d91ddee None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:03.670525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec91c-c270-4811-b18c-c8123d91ddee None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:03.670860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec91c-c270-4811-b18c-c8123d91ddee None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:03.672412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 674/1352] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:03.677675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35a31-0dce-4d3d-883b-56b2fb5f240e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:03.678161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35a31-0dce-4d3d-883b-56b2fb5f240e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:03.678458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35a31-0dce-4d3d-883b-56b2fb5f240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:03.734856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:03.761193 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-15a35a31-0dce-4d3d-883b-56b2fb5f240e None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:03.762530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-15a35a31-0dce-4d3d-883b-56b2fb5f240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:03.878939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 679/1353] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:10:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:05.487027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6bded-01cd-4bb7-bbf7-7d3d09f6a146 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:05.487571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6bded-01cd-4bb7-bbf7-7d3d09f6a146 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:05.487571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6bded-01cd-4bb7-bbf7-7d3d09f6a146 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:05.488972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 675/1354] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:05.494599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8500-ac95-4e47-bcbb-aca017d56a45 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:05.495167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8500-ac95-4e47-bcbb-aca017d56a45 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:05.495167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8500-ac95-4e47-bcbb-aca017d56a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:05.587254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e47e8500-ac95-4e47-bcbb-aca017d56a45 None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:05.589425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e47e8500-ac95-4e47-bcbb-aca017d56a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:05.635221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:05.706913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 680/1355] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:10:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:10:09.016286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3b0d-b12d-40c4-9c3c-a8fa86c5f218 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.017180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3b0d-b12d-40c4-9c3c-a8fa86c5f218 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.017344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3b0d-b12d-40c4-9c3c-a8fa86c5f218 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.018732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 676/1356] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:09.024147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe575e-683c-4e25-9a31-619e45e82375 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.024589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe575e-683c-4e25-9a31-619e45e82375 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.024743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe575e-683c-4e25-9a31-619e45e82375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.096792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-08fe575e-683c-4e25-9a31-619e45e82375 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:09.098740 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-08fe575e-683c-4e25-9a31-619e45e82375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:09.228631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 681/1357] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:09.541613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97f6b8-85a0-47b0-bf67-915af74f0233 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.542310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97f6b8-85a0-47b0-bf67-915af74f0233 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.544650 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97f6b8-85a0-47b0-bf67-915af74f0233 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.545583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 677/1358] 10.222.0.95 () {54 vars in 962 bytes} [Wed Aug 31 19:10:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:10:09.555168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165a676-ec60-4174-98cf-13ae1bd7c031 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.556236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165a676-ec60-4174-98cf-13ae1bd7c031 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.557147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165a676-ec60-4174-98cf-13ae1bd7c031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.581489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:09.647667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8165a676-ec60-4174-98cf-13ae1bd7c031 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:09.649015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8165a676-ec60-4174-98cf-13ae1bd7c031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:09.776235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 682/1359] 10.222.0.95 () {58 vars in 1059 bytes} [Wed Aug 31 19:10:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:09.784902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97f6b8-85a0-47b0-bf67-915af74f0233 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:09.786747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-de97f6b8-85a0-47b0-bf67-915af74f0233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:09.938686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97f6b8-85a0-47b0-bf67-915af74f0233 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:10.016140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:10.059407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97651843-5875-4051-80cd-8cfe9ff4477f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:10.060048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97651843-5875-4051-80cd-8cfe9ff4477f service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:10.060325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97651843-5875-4051-80cd-8cfe9ff4477f service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:10.060627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97651843-5875-4051-80cd-8cfe9ff4477f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:10.062225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-97651843-5875-4051-80cd-8cfe9ff4477f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:10.168083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97651843-5875-4051-80cd-8cfe9ff4477f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:10.170298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97651843-5875-4051-80cd-8cfe9ff4477f service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:10.190219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 678/1360] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:10:12.347398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8635b5e-89c2-488f-b196-28a8cd8b143f None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:12.347908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8635b5e-89c2-488f-b196-28a8cd8b143f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:12.347908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8635b5e-89c2-488f-b196-28a8cd8b143f None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:12.349244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 683/1361] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:12.355056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2812a1-784c-4414-a6a7-86c997e7c80e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:12.355473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2812a1-784c-4414-a6a7-86c997e7c80e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:12.355745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2812a1-784c-4414-a6a7-86c997e7c80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:12.435611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ae2812a1-784c-4414-a6a7-86c997e7c80e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:12.436833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ae2812a1-784c-4414-a6a7-86c997e7c80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:12.479699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:12.549031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 679/1362] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:12.862426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8635b5e-89c2-488f-b196-28a8cd8b143f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:12.862426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a8635b5e-89c2-488f-b196-28a8cd8b143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:12.956191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:13.046903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8635b5e-89c2-488f-b196-28a8cd8b143f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:13.187001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85d94f-4243-48dd-adb5-8530eb7bb8e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:13.187611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85d94f-4243-48dd-adb5-8530eb7bb8e9 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:13.187901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85d94f-4243-48dd-adb5-8530eb7bb8e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:13.188233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85d94f-4243-48dd-adb5-8530eb7bb8e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:13.190925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7b85d94f-4243-48dd-adb5-8530eb7bb8e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:13.219693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:13.300617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85d94f-4243-48dd-adb5-8530eb7bb8e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:13.302253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85d94f-4243-48dd-adb5-8530eb7bb8e9 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:13.321816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 684/1363] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:15.612288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca309eb-c0ae-4bf8-be9c-8c9d397a8afe None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:15.612979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca309eb-c0ae-4bf8-be9c-8c9d397a8afe None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:15.612979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca309eb-c0ae-4bf8-be9c-8c9d397a8afe None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:15.614228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 680/1364] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:15.619776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5370d93-6f97-498c-93db-0af1d2606004 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:15.620217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5370d93-6f97-498c-93db-0af1d2606004 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:15.620392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5370d93-6f97-498c-93db-0af1d2606004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:15.697088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f5370d93-6f97-498c-93db-0af1d2606004 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:15.698304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f5370d93-6f97-498c-93db-0af1d2606004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:15.804901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:15.816536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 685/1365] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:16.105018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1dce7-33b4-4789-8733-b9d9f2a90460 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:16.105436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1dce7-33b4-4789-8733-b9d9f2a90460 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:16.105832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1dce7-33b4-4789-8733-b9d9f2a90460 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:16.107296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 681/1366] 10.222.0.95 () {54 vars in 962 bytes} [Wed Aug 31 19:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:16.113914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d233-9e45-4d3c-b672-9a0ba664cac4 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:16.113914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d233-9e45-4d3c-b672-9a0ba664cac4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:16.113914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d233-9e45-4d3c-b672-9a0ba664cac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:16.201000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-2494d233-9e45-4d3c-b672-9a0ba664cac4 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:16.202779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2494d233-9e45-4d3c-b672-9a0ba664cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:16.338586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 686/1367] 10.222.0.95 () {58 vars in 1059 bytes} [Wed Aug 31 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:16.347762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1dce7-33b4-4789-8733-b9d9f2a90460 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:16.349427 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-21a1dce7-33b4-4789-8733-b9d9f2a90460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:16.516119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1dce7-33b4-4789-8733-b9d9f2a90460 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:16.539623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:16.640337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065e37b-680d-49d5-8454-69d0ed260767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:16.641059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e37b-680d-49d5-8454-69d0ed260767 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:16.641285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e37b-680d-49d5-8454-69d0ed260767 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:16.642728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e37b-680d-49d5-8454-69d0ed260767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:16.645065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0065e37b-680d-49d5-8454-69d0ed260767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:16.739605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065e37b-680d-49d5-8454-69d0ed260767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:16.741819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065e37b-680d-49d5-8454-69d0ed260767 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:16.763954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 682/1368] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:17.343560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fcd8c-30ab-4832-80a1-ce7a2386cbb0 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:17.344102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fcd8c-30ab-4832-80a1-ce7a2386cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:17.344102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fcd8c-30ab-4832-80a1-ce7a2386cbb0 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:17.346280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 687/1369] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:17.351719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a1112-8f71-4240-b4e7-25601a5172fa None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:17.352002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a1112-8f71-4240-b4e7-25601a5172fa None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:17.352278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a1112-8f71-4240-b4e7-25601a5172fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:17.386558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:17.427019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8d0a1112-8f71-4240-b4e7-25601a5172fa None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:17.428231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8d0a1112-8f71-4240-b4e7-25601a5172fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:17.535358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 683/1370] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:18.160251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1caf0e-4493-4713-9a12-0a5e9179c429 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:18.160251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1caf0e-4493-4713-9a12-0a5e9179c429 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:18.160251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1caf0e-4493-4713-9a12-0a5e9179c429 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:18.162832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 688/1371] 10.222.0.95 () {54 vars in 964 bytes} [Wed Aug 31 19:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:18.167011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d54b7-83cc-429c-9f65-c1c50075d434 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:18.167501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d54b7-83cc-429c-9f65-c1c50075d434 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:18.167599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d54b7-83cc-429c-9f65-c1c50075d434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:18.254278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fe7d54b7-83cc-429c-9f65-c1c50075d434 None None] MFA Rules not processed for user `115298313c1243d282d09a8e8005002c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:18.256299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fe7d54b7-83cc-429c-9f65-c1c50075d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:18.313353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:18.347786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 684/1372] 10.222.0.95 () {58 vars in 1061 bytes} [Wed Aug 31 19:10:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:18.355254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1caf0e-4493-4713-9a12-0a5e9179c429 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:18.357169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fe1caf0e-4493-4713-9a12-0a5e9179c429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:18.414819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:18.480604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1caf0e-4493-4713-9a12-0a5e9179c429 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:18.564081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627a97b-077a-42eb-b175-b0588e1d4184 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:18.564744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627a97b-077a-42eb-b175-b0588e1d4184 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:18.565213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627a97b-077a-42eb-b175-b0588e1d4184 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:18.565936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627a97b-077a-42eb-b175-b0588e1d4184 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:18.567165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0627a97b-077a-42eb-b175-b0588e1d4184 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:18.691611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:18.703275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627a97b-077a-42eb-b175-b0588e1d4184 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:18.705961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627a97b-077a-42eb-b175-b0588e1d4184 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:18.727890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 689/1373] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:19.002648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d995f-9ec7-4e39-8908-75b3f18d0130 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:19.002931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d995f-9ec7-4e39-8908-75b3f18d0130 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:19.003207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d995f-9ec7-4e39-8908-75b3f18d0130 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:19.004560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 685/1374] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:19.009748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b90f-533e-478b-a86b-b841c98943e5 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:19.010035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b90f-533e-478b-a86b-b841c98943e5 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:19.010313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236b90f-533e-478b-a86b-b841c98943e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:19.079197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6236b90f-533e-478b-a86b-b841c98943e5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:19.081556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6236b90f-533e-478b-a86b-b841c98943e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:19.187694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 690/1375] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:19.485024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d995f-9ec7-4e39-8908-75b3f18d0130 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:19.487152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6c7d995f-9ec7-4e39-8908-75b3f18d0130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:19.634806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d995f-9ec7-4e39-8908-75b3f18d0130 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:19.686754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:19.750735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06f875-462e-4398-9987-e035b05b3458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:19.752086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06f875-462e-4398-9987-e035b05b3458 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:19.752392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06f875-462e-4398-9987-e035b05b3458 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:19.752695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06f875-462e-4398-9987-e035b05b3458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:19.754281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6d06f875-462e-4398-9987-e035b05b3458 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:19.845837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06f875-462e-4398-9987-e035b05b3458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:19.847477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06f875-462e-4398-9987-e035b05b3458 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:19.865542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 686/1376] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:21.863279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7186-a183-4ee6-b9ec-7a277afba03c None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:21.863790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7186-a183-4ee6-b9ec-7a277afba03c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:21.863790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7186-a183-4ee6-b9ec-7a277afba03c None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:21.865119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 691/1377] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:21.870296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437c15-2110-4176-991f-65297ffb7a04 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:21.870759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437c15-2110-4176-991f-65297ffb7a04 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:21.870858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437c15-2110-4176-991f-65297ffb7a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:21.937240 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d4437c15-2110-4176-991f-65297ffb7a04 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:21.938426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d4437c15-2110-4176-991f-65297ffb7a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:21.945669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:22.040233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 687/1378] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:22.357281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba7186-a183-4ee6-b9ec-7a277afba03c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:22.361391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e3ba7186-a183-4ee6-b9ec-7a277afba03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:22.460149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:22.565041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba7186-a183-4ee6-b9ec-7a277afba03c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:22.709348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869471fd-9179-4e99-af57-fcf31d17170b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:22.710140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869471fd-9179-4e99-af57-fcf31d17170b service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:22.710564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869471fd-9179-4e99-af57-fcf31d17170b service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:22.710838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869471fd-9179-4e99-af57-fcf31d17170b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:22.712508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-869471fd-9179-4e99-af57-fcf31d17170b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:22.734549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:22.816120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869471fd-9179-4e99-af57-fcf31d17170b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:22.817764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869471fd-9179-4e99-af57-fcf31d17170b service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:22.838030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 692/1379] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:24.841669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb0c71-099c-4762-8740-3f40d0b74cdd None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:24.842246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb0c71-099c-4762-8740-3f40d0b74cdd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:24.842246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb0c71-099c-4762-8740-3f40d0b74cdd None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:24.843692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 688/1380] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:24.849670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1812cb-228d-4144-bd67-b723c2a402bd None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:24.850140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1812cb-228d-4144-bd67-b723c2a402bd None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:24.850250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1812cb-228d-4144-bd67-b723c2a402bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:24.922698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-3b1812cb-228d-4144-bd67-b723c2a402bd None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:24.924814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3b1812cb-228d-4144-bd67-b723c2a402bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:25.030438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:25.039782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 693/1381] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:25.344484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb0c71-099c-4762-8740-3f40d0b74cdd None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:25.346400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e5cb0c71-099c-4762-8740-3f40d0b74cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:25.501870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:25.514780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb0c71-099c-4762-8740-3f40d0b74cdd None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:25.642715 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7b37e-fd04-4a62-92e2-489b4960cc09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:25.643250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b37e-fd04-4a62-92e2-489b4960cc09 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:25.643606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b37e-fd04-4a62-92e2-489b4960cc09 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:25.643829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7b37e-fd04-4a62-92e2-489b4960cc09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:25.645465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a2b7b37e-fd04-4a62-92e2-489b4960cc09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:25.754323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7b37e-fd04-4a62-92e2-489b4960cc09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:25.755950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7b37e-fd04-4a62-92e2-489b4960cc09 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:25.767556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:25.777453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 689/1382] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:27.752702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa2c2d-a0d1-443e-a854-ba9666cc61be None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:27.753263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa2c2d-a0d1-443e-a854-ba9666cc61be None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:27.753263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa2c2d-a0d1-443e-a854-ba9666cc61be None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:27.754602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 694/1383] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:27.760087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e640d-1087-4d17-af84-ebb636f8e37d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:27.760694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e640d-1087-4d17-af84-ebb636f8e37d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:27.760694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e640d-1087-4d17-af84-ebb636f8e37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:27.831379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b59e640d-1087-4d17-af84-ebb636f8e37d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:27.832672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b59e640d-1087-4d17-af84-ebb636f8e37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:27.979863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 690/1384] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:10:28.277049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa2c2d-a0d1-443e-a854-ba9666cc61be None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:28.278588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d7aa2c2d-a0d1-443e-a854-ba9666cc61be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:28.447685 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa2c2d-a0d1-443e-a854-ba9666cc61be None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:28.524416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:28.581813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d63c1-b565-4374-94b4-63ca334e27d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:28.582594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d63c1-b565-4374-94b4-63ca334e27d5 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:28.582954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d63c1-b565-4374-94b4-63ca334e27d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:28.583222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d63c1-b565-4374-94b4-63ca334e27d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:28.585242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-104d63c1-b565-4374-94b4-63ca334e27d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:28.685309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d63c1-b565-4374-94b4-63ca334e27d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:28.687257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d63c1-b565-4374-94b4-63ca334e27d5 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:28.707130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 695/1385] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:30.974610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e716-a411-4e4a-bd3d-7bcbed1ff7ab None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:30.975235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e716-a411-4e4a-bd3d-7bcbed1ff7ab None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:30.975342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e716-a411-4e4a-bd3d-7bcbed1ff7ab None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:30.976999 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 691/1386] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:30.983153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466bf84-662c-4600-ab54-0182ba629a6f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:30.983621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466bf84-662c-4600-ab54-0182ba629a6f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:30.983725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466bf84-662c-4600-ab54-0182ba629a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:31.056896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-9466bf84-662c-4600-ab54-0182ba629a6f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:31.058182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9466bf84-662c-4600-ab54-0182ba629a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:31.118608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:31.223249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 696/1387] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:10:31.525184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724e716-a411-4e4a-bd3d-7bcbed1ff7ab None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:31.526716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8724e716-a411-4e4a-bd3d-7bcbed1ff7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:31.614067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:31.694722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724e716-a411-4e4a-bd3d-7bcbed1ff7ab None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:31.817361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cd129-62c9-4e35-ada5-0f1add5219a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:31.817888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cd129-62c9-4e35-ada5-0f1add5219a4 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:31.818136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cd129-62c9-4e35-ada5-0f1add5219a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:31.818411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cd129-62c9-4e35-ada5-0f1add5219a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:31.820028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5e5cd129-62c9-4e35-ada5-0f1add5219a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:31.860956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:31.917298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cd129-62c9-4e35-ada5-0f1add5219a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:31.919195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cd129-62c9-4e35-ada5-0f1add5219a4 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:31.944432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 692/1388] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:33.914642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52500ee6-7f23-422f-96f1-d59fcacca7b0 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:33.915265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52500ee6-7f23-422f-96f1-d59fcacca7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:33.915265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52500ee6-7f23-422f-96f1-d59fcacca7b0 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:33.916529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 697/1389] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:33.921901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380033f-0a33-4119-bb82-a83bc47b0dd4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:33.922361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380033f-0a33-4119-bb82-a83bc47b0dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:33.922459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380033f-0a33-4119-bb82-a83bc47b0dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:33.988775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9865e-1c0e-4216-bb89-7560b5ab3318 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:33.989032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9865e-1c0e-4216-bb89-7560b5ab3318 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:33.989754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9865e-1c0e-4216-bb89-7560b5ab3318 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:33.991154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 698/1390] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:10:33.991894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c380033f-0a33-4119-bb82-a83bc47b0dd4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:33.993143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c380033f-0a33-4119-bb82-a83bc47b0dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:33.996290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2e7e7-4de1-4290-b51c-37b3f78f81d7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:33.996675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2e7e7-4de1-4290-b51c-37b3f78f81d7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:33.996962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2e7e7-4de1-4290-b51c-37b3f78f81d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:34.066974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c9f2e7e7-4de1-4290-b51c-37b3f78f81d7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:34.068302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c9f2e7e7-4de1-4290-b51c-37b3f78f81d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:34.092801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 693/1391] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:34.168265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 699/1392] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:34.382871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380033f-0a33-4119-bb82-a83bc47b0dd4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:34.384128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c380033f-0a33-4119-bb82-a83bc47b0dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:34.398356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:34.475343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0898d8-64b3-4434-b510-2afb916171cc None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:34.475634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0898d8-64b3-4434-b510-2afb916171cc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:34.475907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0898d8-64b3-4434-b510-2afb916171cc None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:34.477301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 700/1393] 10.222.0.95 () {54 vars in 967 bytes} [Wed Aug 31 19:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:10:34.481964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18745f76-30db-4a51-a21c-2f7a7a44daa8 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:34.482246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18745f76-30db-4a51-a21c-2f7a7a44daa8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:34.482522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18745f76-30db-4a51-a21c-2f7a7a44daa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:34.499279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:34.501736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380033f-0a33-4119-bb82-a83bc47b0dd4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:34.569241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-18745f76-30db-4a51-a21c-2f7a7a44daa8 None None] MFA Rules not processed for user `8b6c90a2b95c498494a73b43e7138905`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:34.570552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-18745f76-30db-4a51-a21c-2f7a7a44daa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:34.624478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92091a0-8e4e-493b-ae55-7a23a1663c8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:34.625031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92091a0-8e4e-493b-ae55-7a23a1663c8b service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:34.625315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92091a0-8e4e-493b-ae55-7a23a1663c8b service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:34.625613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92091a0-8e4e-493b-ae55-7a23a1663c8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:34.627336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d92091a0-8e4e-493b-ae55-7a23a1663c8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:34.636042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:34.650989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 701/1394] 10.222.0.95 () {58 vars in 1064 bytes} [Wed Aug 31 19:10:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:10:34.659428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18745f76-30db-4a51-a21c-2f7a7a44daa8 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:34.667214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18745f76-30db-4a51-a21c-2f7a7a44daa8 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:34.720397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92091a0-8e4e-493b-ae55-7a23a1663c8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:34.722051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92091a0-8e4e-493b-ae55-7a23a1663c8b service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:34.740697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 694/1395] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:34.743520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f222e-aca9-4bf7-ab0b-21be749bee12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6c90a2b95c498494a73b43e7138905', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:34.744155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f222e-aca9-4bf7-ab0b-21be749bee12 service glance] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:34.744434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f222e-aca9-4bf7-ab0b-21be749bee12 service glance] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:34.744743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f222e-aca9-4bf7-ab0b-21be749bee12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:34.746508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ba1f222e-aca9-4bf7-ab0b-21be749bee12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:34.778250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:34.878315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f222e-aca9-4bf7-ab0b-21be749bee12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:34.879714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f222e-aca9-4bf7-ab0b-21be749bee12 service glance] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:34.902801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 702/1396] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:10:35.695733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc0ba-6ee9-4402-bbf1-aa5e0a56941a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:35.696238 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc0ba-6ee9-4402-bbf1-aa5e0a56941a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:35.696342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc0ba-6ee9-4402-bbf1-aa5e0a56941a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:35.764717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cbffc0ba-6ee9-4402-bbf1-aa5e0a56941a None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:35.765932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cbffc0ba-6ee9-4402-bbf1-aa5e0a56941a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:35.851461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:35.855016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 695/1397] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:10:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:35.902945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858e2c-a119-4e19-af36-8b96f86b60df None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:35.903458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858e2c-a119-4e19-af36-8b96f86b60df None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:35.903556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858e2c-a119-4e19-af36-8b96f86b60df None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:35.904909 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 703/1398] 10.222.0.95 () {54 vars in 943 bytes} [Wed Aug 31 19:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:35.911651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e1e8b-7a6b-4110-bf1a-4f562a810d97 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:35.911943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e1e8b-7a6b-4110-bf1a-4f562a810d97 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:35.912246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e1e8b-7a6b-4110-bf1a-4f562a810d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:35.990526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0b6e1e8b-7a6b-4110-bf1a-4f562a810d97 None None] MFA Rules not processed for user `bac9441fcf60412dbc33e0589a86c8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:35.991723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0b6e1e8b-7a6b-4110-bf1a-4f562a810d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:36.099907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 696/1399] 10.222.0.95 () {58 vars in 1040 bytes} [Wed Aug 31 19:10:35 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:36.112885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b858e2c-a119-4e19-af36-8b96f86b60df None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:36.114522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6b858e2c-a119-4e19-af36-8b96f86b60df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:36.257188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:36.260132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b858e2c-a119-4e19-af36-8b96f86b60df None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:36.367684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ede03-35a2-4ffa-a5da-6f4a60551ded service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:36.368311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ede03-35a2-4ffa-a5da-6f4a60551ded service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:36.368612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ede03-35a2-4ffa-a5da-6f4a60551ded service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:36.368884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ede03-35a2-4ffa-a5da-6f4a60551ded service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:36.370569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-372ede03-35a2-4ffa-a5da-6f4a60551ded service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:36.459135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ede03-35a2-4ffa-a5da-6f4a60551ded service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:36.460697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ede03-35a2-4ffa-a5da-6f4a60551ded service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:36.470196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 704/1400] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:10:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:10:36.866861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca15d6e-1d7b-4f5b-9cec-900f1c90c9ca None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:36.866861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca15d6e-1d7b-4f5b-9cec-900f1c90c9ca None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:36.867870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca15d6e-1d7b-4f5b-9cec-900f1c90c9ca None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:36.869720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 697/1401] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:36.876365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19efbf-5c46-45b6-bcc4-959847a7ae98 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:36.878204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19efbf-5c46-45b6-bcc4-959847a7ae98 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:36.878204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19efbf-5c46-45b6-bcc4-959847a7ae98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:36.933939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:36.973252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6c19efbf-5c46-45b6-bcc4-959847a7ae98 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:36.974510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6c19efbf-5c46-45b6-bcc4-959847a7ae98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:37.090928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 705/1402] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:10:37.400045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca15d6e-1d7b-4f5b-9cec-900f1c90c9ca None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:37.401999 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fca15d6e-1d7b-4f5b-9cec-900f1c90c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:37.472940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:37.584543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca15d6e-1d7b-4f5b-9cec-900f1c90c9ca None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:37.765455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50000c85-917a-491f-8344-82bd07f36122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:37.766067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50000c85-917a-491f-8344-82bd07f36122 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:37.766632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50000c85-917a-491f-8344-82bd07f36122 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:37.766772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50000c85-917a-491f-8344-82bd07f36122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:37.768310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-50000c85-917a-491f-8344-82bd07f36122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:37.790508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:37.873946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50000c85-917a-491f-8344-82bd07f36122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:37.876327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50000c85-917a-491f-8344-82bd07f36122 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:37.896380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 698/1403] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:39.894799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef961b-1429-4113-b9da-480ade2e2d5b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:39.895416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef961b-1429-4113-b9da-480ade2e2d5b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:39.895416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef961b-1429-4113-b9da-480ade2e2d5b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:39.896908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 706/1404] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:39.902141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f223c5-4828-45ad-800d-cfbef32755d6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:39.902618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f223c5-4828-45ad-800d-cfbef32755d6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:39.902719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f223c5-4828-45ad-800d-cfbef32755d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:39.981977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d3f223c5-4828-45ad-800d-cfbef32755d6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:39.983142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d3f223c5-4828-45ad-800d-cfbef32755d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:40.090019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:40.100433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 699/1405] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:40.399941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef961b-1429-4113-b9da-480ade2e2d5b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:40.401558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a1ef961b-1429-4113-b9da-480ade2e2d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:40.538366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:40.588198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef961b-1429-4113-b9da-480ade2e2d5b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:40.725997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23dba4-4688-43b8-826e-05f28570804e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:40.726703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23dba4-4688-43b8-826e-05f28570804e service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:40.726978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23dba4-4688-43b8-826e-05f28570804e service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:40.727244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23dba4-4688-43b8-826e-05f28570804e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:40.730619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-dd23dba4-4688-43b8-826e-05f28570804e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:40.811491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:40.835007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23dba4-4688-43b8-826e-05f28570804e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:40.836654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23dba4-4688-43b8-826e-05f28570804e service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:40.856549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 707/1406] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:42.959938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98567e8-ed34-4f6f-9ba8-bf57b35f326b None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:42.960983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98567e8-ed34-4f6f-9ba8-bf57b35f326b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:42.961499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98567e8-ed34-4f6f-9ba8-bf57b35f326b None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:42.963100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 700/1407] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:10:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:42.968509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4722b20-3d67-475a-b1a9-a269d8211e39 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:42.969043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4722b20-3d67-475a-b1a9-a269d8211e39 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:42.969145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4722b20-3d67-475a-b1a9-a269d8211e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:43.049214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e4722b20-3d67-475a-b1a9-a269d8211e39 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:43.050442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e4722b20-3d67-475a-b1a9-a269d8211e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:43.171865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 708/1408] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:43.467797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98567e8-ed34-4f6f-9ba8-bf57b35f326b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:43.470056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c98567e8-ed34-4f6f-9ba8-bf57b35f326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:43.641680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98567e8-ed34-4f6f-9ba8-bf57b35f326b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:43.723752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:43.779586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bda29e-ee09-4982-8680-8de1ffb77f87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:43.780208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bda29e-ee09-4982-8680-8de1ffb77f87 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:43.780493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bda29e-ee09-4982-8680-8de1ffb77f87 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:43.780797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bda29e-ee09-4982-8680-8de1ffb77f87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:43.783689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-32bda29e-ee09-4982-8680-8de1ffb77f87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:43.906621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bda29e-ee09-4982-8680-8de1ffb77f87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:43.908437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bda29e-ee09-4982-8680-8de1ffb77f87 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:43.930883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 701/1409] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:53.170210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=79 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:53.928185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=64 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:58.712622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe417a-c144-47af-9a05-289d39dfdf03 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:58.713196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe417a-c144-47af-9a05-289d39dfdf03 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:58.713196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe417a-c144-47af-9a05-289d39dfdf03 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:58.714516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 709/1410] 10.222.0.95 () {54 vars in 918 bytes} [Wed Aug 31 19:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:58.720499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e112f-7d51-490b-bd15-efb1e9351e81 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:58.721075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e112f-7d51-490b-bd15-efb1e9351e81 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:58.721075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e112f-7d51-490b-bd15-efb1e9351e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:58.803286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f14e112f-7d51-490b-bd15-efb1e9351e81 None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:58.804423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f14e112f-7d51-490b-bd15-efb1e9351e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:58.915065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 702/1411] 10.222.0.95 () {58 vars in 1015 bytes} [Wed Aug 31 19:10:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:58.930328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb7ad-4fea-4c97-ba47-4b5df85e0af3 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:58.930611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb7ad-4fea-4c97-ba47-4b5df85e0af3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:58.930888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fb7ad-4fea-4c97-ba47-4b5df85e0af3 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:58.932252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 710/1412] 10.222.0.95 () {54 vars in 918 bytes} [Wed Aug 31 19:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:58.937200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e34ad-f045-4926-bab4-1495e21d75f0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:58.938440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e34ad-f045-4926-bab4-1495e21d75f0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:58.938716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e34ad-f045-4926-bab4-1495e21d75f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:59.020514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:59.026660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0a4e34ad-f045-4926-bab4-1495e21d75f0 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:59.026660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0a4e34ad-f045-4926-bab4-1495e21d75f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:59.153576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 703/1413] 10.222.0.95 () {58 vars in 1015 bytes} [Wed Aug 31 19:10:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:11:01.599764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a532-bb94-43d0-ab9f-e3276b88453a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:01.600323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a532-bb94-43d0-ab9f-e3276b88453a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:01.600323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5a532-bb94-43d0-ab9f-e3276b88453a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:01.601763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:01.607216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee50b-5cf2-4c7d-ad96-22eeff65445e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:01.607756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee50b-5cf2-4c7d-ad96-22eeff65445e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:01.608085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee50b-5cf2-4c7d-ad96-22eeff65445e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:01.678789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f52ee50b-5cf2-4c7d-ad96-22eeff65445e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:01.680002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f52ee50b-5cf2-4c7d-ad96-22eeff65445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:01.737005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:01.780135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:02.083904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5a532-bb94-43d0-ab9f-e3276b88453a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:02.085518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-12f5a532-bb94-43d0-ab9f-e3276b88453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:02.236285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5a532-bb94-43d0-ab9f-e3276b88453a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:02.307405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:02.349131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c035fc2-41e0-4ca7-a5c6-a9934dc65ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:02.349661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c035fc2-41e0-4ca7-a5c6-a9934dc65ee5 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:02.349943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c035fc2-41e0-4ca7-a5c6-a9934dc65ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:02.350218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c035fc2-41e0-4ca7-a5c6-a9934dc65ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:02.351831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6c035fc2-41e0-4ca7-a5c6-a9934dc65ee5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:02.440627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c035fc2-41e0-4ca7-a5c6-a9934dc65ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:02.442246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c035fc2-41e0-4ca7-a5c6-a9934dc65ee5 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:02.460707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 712/1416] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:03.360108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdf3af-f467-48a9-a287-4721daca4bb0 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:03.360790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdf3af-f467-48a9-a287-4721daca4bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:03.360790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdf3af-f467-48a9-a287-4721daca4bb0 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:03.362023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:03.366943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e94aa-b028-49f0-8b8e-5c288eb9fbc2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:03.367343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e94aa-b028-49f0-8b8e-5c288eb9fbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:03.367504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e94aa-b028-49f0-8b8e-5c288eb9fbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:03.443737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-cc3e94aa-b028-49f0-8b8e-5c288eb9fbc2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:03.444908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cc3e94aa-b028-49f0-8b8e-5c288eb9fbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:03.471575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:03.545439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:03.701535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdf3af-f467-48a9-a287-4721daca4bb0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:03.703592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a8cdf3af-f467-48a9-a287-4721daca4bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:03.852915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdf3af-f467-48a9-a287-4721daca4bb0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:03.893762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:03.966714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54744f92-d3d0-4f90-9119-8fd9f365a6f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:03.967321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54744f92-d3d0-4f90-9119-8fd9f365a6f9 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:03.967602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54744f92-d3d0-4f90-9119-8fd9f365a6f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:03.967870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54744f92-d3d0-4f90-9119-8fd9f365a6f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:03.969553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-54744f92-d3d0-4f90-9119-8fd9f365a6f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:04.058942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54744f92-d3d0-4f90-9119-8fd9f365a6f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:04.060544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54744f92-d3d0-4f90-9119-8fd9f365a6f9 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:04.078803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 706/1419] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:04.964150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ef6c-a1d0-4ae5-92ed-b65cbcd6d89e None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:04.964755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ef6c-a1d0-4ae5-92ed-b65cbcd6d89e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:04.964755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ef6c-a1d0-4ae5-92ed-b65cbcd6d89e None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:04.966088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:04.971285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b85fa4-10db-4275-85e3-736706dd0d4a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:04.971740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b85fa4-10db-4275-85e3-736706dd0d4a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:04.971840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b85fa4-10db-4275-85e3-736706dd0d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:05.026630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:05.050766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-72b85fa4-10db-4275-85e3-736706dd0d4a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:05.051982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-72b85fa4-10db-4275-85e3-736706dd0d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:05.149038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:05.304393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ef6c-a1d0-4ae5-92ed-b65cbcd6d89e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:05.305854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-17b3ef6c-a1d0-4ae5-92ed-b65cbcd6d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:05.450131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ef6c-a1d0-4ae5-92ed-b65cbcd6d89e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:05.462782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:05.563829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13515a15-41a1-4cef-881f-547f27d02720 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:05.564334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515a15-41a1-4cef-881f-547f27d02720 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:05.564638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515a15-41a1-4cef-881f-547f27d02720 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:05.564915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515a15-41a1-4cef-881f-547f27d02720 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:05.566518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-13515a15-41a1-4cef-881f-547f27d02720 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:05.657329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13515a15-41a1-4cef-881f-547f27d02720 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:05.658959 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13515a15-41a1-4cef-881f-547f27d02720 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:05.678163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 715/1422] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:06.552336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2babf-b409-4055-9b94-fc235643c9c2 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:06.552868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2babf-b409-4055-9b94-fc235643c9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:06.553000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2babf-b409-4055-9b94-fc235643c9c2 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:06.554249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:06.559050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9405b-45de-4851-93c5-716f7e55448b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:06.559442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9405b-45de-4851-93c5-716f7e55448b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:06.559603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9405b-45de-4851-93c5-716f7e55448b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:06.580996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:06.638806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4ae9405b-45de-4851-93c5-716f7e55448b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:06.639997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4ae9405b-45de-4851-93c5-716f7e55448b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:06.735163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:06.887668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2babf-b409-4055-9b94-fc235643c9c2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:06.889223 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3eb2babf-b409-4055-9b94-fc235643c9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:06.998970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:07.037470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2babf-b409-4055-9b94-fc235643c9c2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:07.149497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de5bcd-68e4-4bab-8c6d-815793ea9013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:07.150034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de5bcd-68e4-4bab-8c6d-815793ea9013 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:07.150308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de5bcd-68e4-4bab-8c6d-815793ea9013 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:07.150575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de5bcd-68e4-4bab-8c6d-815793ea9013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:07.152148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-87de5bcd-68e4-4bab-8c6d-815793ea9013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:07.229431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:07.246236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de5bcd-68e4-4bab-8c6d-815793ea9013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:07.247847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de5bcd-68e4-4bab-8c6d-815793ea9013 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:07.265840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 709/1425] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:08.130571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ea0dc-f196-4457-840d-2e517fef4a88 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:08.131005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ea0dc-f196-4457-840d-2e517fef4a88 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:08.131160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ea0dc-f196-4457-840d-2e517fef4a88 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:08.132544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:08.137559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad43e8-9d78-4ba7-baa9-f60b61c4c2e7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:08.138029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad43e8-9d78-4ba7-baa9-f60b61c4c2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:08.138130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad43e8-9d78-4ba7-baa9-f60b61c4c2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:08.215419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ebad43e8-9d78-4ba7-baa9-f60b61c4c2e7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:08.216647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ebad43e8-9d78-4ba7-baa9-f60b61c4c2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:08.313933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:11:08.471482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ea0dc-f196-4457-840d-2e517fef4a88 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:08.473022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-da1ea0dc-f196-4457-840d-2e517fef4a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:08.559198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:08.623262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ea0dc-f196-4457-840d-2e517fef4a88 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:08.738905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26b0f2-7d1b-405c-8b6d-140c5feae756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:08.739430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26b0f2-7d1b-405c-8b6d-140c5feae756 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:08.739707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26b0f2-7d1b-405c-8b6d-140c5feae756 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:08.739976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26b0f2-7d1b-405c-8b6d-140c5feae756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:08.741628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9b26b0f2-7d1b-405c-8b6d-140c5feae756 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:08.791682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:08.834144 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26b0f2-7d1b-405c-8b6d-140c5feae756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:08.835780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26b0f2-7d1b-405c-8b6d-140c5feae756 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:08.853679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 718/1428] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:11:09.694973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa668a-1400-458b-99a9-4206871fc714 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:09.695568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa668a-1400-458b-99a9-4206871fc714 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:09.695568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa668a-1400-458b-99a9-4206871fc714 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:09.696895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:09.702107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0358ec-114d-49eb-a3c9-f0a9e4f98adf None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:09.702575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0358ec-114d-49eb-a3c9-f0a9e4f98adf None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:09.702680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0358ec-114d-49eb-a3c9-f0a9e4f98adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:09.781639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8c0358ec-114d-49eb-a3c9-f0a9e4f98adf None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:09.782846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8c0358ec-114d-49eb-a3c9-f0a9e4f98adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:09.883562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:10.037364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa668a-1400-458b-99a9-4206871fc714 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:10.038953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e3aa668a-1400-458b-99a9-4206871fc714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:10.088048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:10.185284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa668a-1400-458b-99a9-4206871fc714 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:10.299300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42807b2e-6ce7-4d9f-a4f4-676549e79319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:10.299819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42807b2e-6ce7-4d9f-a4f4-676549e79319 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:10.300098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42807b2e-6ce7-4d9f-a4f4-676549e79319 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:10.300369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42807b2e-6ce7-4d9f-a4f4-676549e79319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:10.302008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-42807b2e-6ce7-4d9f-a4f4-676549e79319 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:10.308944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:10.394423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42807b2e-6ce7-4d9f-a4f4-676549e79319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:10.396055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42807b2e-6ce7-4d9f-a4f4-676549e79319 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:10.414327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 712/1431] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:11.247980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aaa15-915c-4f84-a8f5-063092b8e608 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:11.248561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aaa15-915c-4f84-a8f5-063092b8e608 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:11.248561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aaa15-915c-4f84-a8f5-063092b8e608 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:11.250699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:11.255861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb495a-d08b-40e5-8704-2f20036df378 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:11.256296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb495a-d08b-40e5-8704-2f20036df378 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:11.256443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb495a-d08b-40e5-8704-2f20036df378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:11.339182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4eeb495a-d08b-40e5-8704-2f20036df378 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:11.340314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4eeb495a-d08b-40e5-8704-2f20036df378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:11.406514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:11.436577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:11.594417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705aaa15-915c-4f84-a8f5-063092b8e608 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:11.595953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-705aaa15-915c-4f84-a8f5-063092b8e608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:11.613852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:11.746107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705aaa15-915c-4f84-a8f5-063092b8e608 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:11.836390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:11.863894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4414a09-40fa-4540-898c-ab42fa13790c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:11.864423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414a09-40fa-4540-898c-ab42fa13790c service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:11.864739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414a09-40fa-4540-898c-ab42fa13790c service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:11.865014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414a09-40fa-4540-898c-ab42fa13790c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:11.866625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b4414a09-40fa-4540-898c-ab42fa13790c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:11.956895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4414a09-40fa-4540-898c-ab42fa13790c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:11.958538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4414a09-40fa-4540-898c-ab42fa13790c service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:11.976876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 721/1434] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:12.791943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b72a6-9b73-4ebe-85b5-178105f6b194 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:12.792825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b72a6-9b73-4ebe-85b5-178105f6b194 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:12.792825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b72a6-9b73-4ebe-85b5-178105f6b194 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:12.794158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:12.799198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c4420-7040-4811-a5eb-46dc7cc8e69f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:12.799688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c4420-7040-4811-a5eb-46dc7cc8e69f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:12.799795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c4420-7040-4811-a5eb-46dc7cc8e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:12.876723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d20c4420-7040-4811-a5eb-46dc7cc8e69f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:12.877923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d20c4420-7040-4811-a5eb-46dc7cc8e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:12.919567 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:12.976963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:13.127749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b72a6-9b73-4ebe-85b5-178105f6b194 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:13.129415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0b4b72a6-9b73-4ebe-85b5-178105f6b194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:13.296552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b72a6-9b73-4ebe-85b5-178105f6b194 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:13.359734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:13.420191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a3174-a8ff-49de-a64b-509470ca59a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:13.420745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a3174-a8ff-49de-a64b-509470ca59a1 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:13.421025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a3174-a8ff-49de-a64b-509470ca59a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:13.421321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a3174-a8ff-49de-a64b-509470ca59a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:13.422948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-072a3174-a8ff-49de-a64b-509470ca59a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:13.512345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a3174-a8ff-49de-a64b-509470ca59a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:13.514137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a3174-a8ff-49de-a64b-509470ca59a1 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:13.532944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 715/1437] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:14.339709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad0192-9fe1-4d9c-a683-8f4d0f8e36f2 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:14.340262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad0192-9fe1-4d9c-a683-8f4d0f8e36f2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:14.340262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad0192-9fe1-4d9c-a683-8f4d0f8e36f2 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:14.341564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:14.347105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bb9ad-198a-4a62-b519-d95fe5a72293 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:14.347561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bb9ad-198a-4a62-b519-d95fe5a72293 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:14.347675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bb9ad-198a-4a62-b519-d95fe5a72293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:14.425603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8d4bb9ad-198a-4a62-b519-d95fe5a72293 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:14.426824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8d4bb9ad-198a-4a62-b519-d95fe5a72293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:14.428858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:14.526934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:14.677551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad0192-9fe1-4d9c-a683-8f4d0f8e36f2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:14.678993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5dad0192-9fe1-4d9c-a683-8f4d0f8e36f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:14.826000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad0192-9fe1-4d9c-a683-8f4d0f8e36f2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:14.852483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:14.939392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55958106-f65e-47e0-9a0a-0414516b4c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:14.939912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55958106-f65e-47e0-9a0a-0414516b4c64 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:14.940190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55958106-f65e-47e0-9a0a-0414516b4c64 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:14.940462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55958106-f65e-47e0-9a0a-0414516b4c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:14.942106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-55958106-f65e-47e0-9a0a-0414516b4c64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:15.030137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55958106-f65e-47e0-9a0a-0414516b4c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:15.032651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55958106-f65e-47e0-9a0a-0414516b4c64 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:15.053298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 724/1440] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:16.914143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ad9be-3f70-4684-9676-315e18efa55f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:16.914730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ad9be-3f70-4684-9676-315e18efa55f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:16.914730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ad9be-3f70-4684-9676-315e18efa55f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:16.916180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:16.922349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36d55-abca-4e48-ae49-4206d0a75e61 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:16.922808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36d55-abca-4e48-ae49-4206d0a75e61 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:16.923063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36d55-abca-4e48-ae49-4206d0a75e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:16.962776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:16.989380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-92d36d55-abca-4e48-ae49-4206d0a75e61 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:16.990605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-92d36d55-abca-4e48-ae49-4206d0a75e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:17.087437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:17.380455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ad9be-3f70-4684-9676-315e18efa55f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:17.382015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0d8ad9be-3f70-4684-9676-315e18efa55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:17.484673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ad9be-3f70-4684-9676-315e18efa55f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:17.518770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:17.559561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc7e9d-216f-4f84-bf1f-2c37caa74885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:17.560166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7e9d-216f-4f84-bf1f-2c37caa74885 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:17.560448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7e9d-216f-4f84-bf1f-2c37caa74885 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:17.560752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7e9d-216f-4f84-bf1f-2c37caa74885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:17.563004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-19fc7e9d-216f-4f84-bf1f-2c37caa74885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:17.673755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc7e9d-216f-4f84-bf1f-2c37caa74885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:17.675200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc7e9d-216f-4f84-bf1f-2c37caa74885 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:17.694227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 718/1443] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:19.779196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c80f4-29fa-453d-8468-a7abf79569dd None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:19.779719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c80f4-29fa-453d-8468-a7abf79569dd None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:19.779815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c80f4-29fa-453d-8468-a7abf79569dd None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:19.781702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:19.787031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cac7-a49f-4bbe-b4bd-2957713841db None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:19.787455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cac7-a49f-4bbe-b4bd-2957713841db None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:19.787586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cac7-a49f-4bbe-b4bd-2957713841db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:19.861245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f158cac7-a49f-4bbe-b4bd-2957713841db None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:19.862463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f158cac7-a49f-4bbe-b4bd-2957713841db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:19.875665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:19.967919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:20.266406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c80f4-29fa-453d-8468-a7abf79569dd None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:20.268019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c49c80f4-29fa-453d-8468-a7abf79569dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:20.371533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c80f4-29fa-453d-8468-a7abf79569dd None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:20.383798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:20.446314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f347a-5a21-4ecd-bde9-513bec625f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:20.446914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f347a-5a21-4ecd-bde9-513bec625f3b service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:20.447191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f347a-5a21-4ecd-bde9-513bec625f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:20.447461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f347a-5a21-4ecd-bde9-513bec625f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:20.449095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cd0f347a-5a21-4ecd-bde9-513bec625f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:20.558453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f347a-5a21-4ecd-bde9-513bec625f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:20.559934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f347a-5a21-4ecd-bde9-513bec625f3b service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:20.579645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 727/1446] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:22.861061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f74fc8-f824-48c0-a6ad-0091cfe952e5 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:22.861716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f74fc8-f824-48c0-a6ad-0091cfe952e5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:22.861716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f74fc8-f824-48c0-a6ad-0091cfe952e5 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:22.862961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:22.868440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d133cc4-c106-4828-9ab4-1d0c2f40c61a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:22.868939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d133cc4-c106-4828-9ab4-1d0c2f40c61a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:22.869038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d133cc4-c106-4828-9ab4-1d0c2f40c61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:22.918527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:22.939542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-9d133cc4-c106-4828-9ab4-1d0c2f40c61a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:22.941049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9d133cc4-c106-4828-9ab4-1d0c2f40c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:23.044145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:23.330500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f74fc8-f824-48c0-a6ad-0091cfe952e5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:23.332023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-24f74fc8-f824-48c0-a6ad-0091cfe952e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:23.440110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f74fc8-f824-48c0-a6ad-0091cfe952e5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:23.481339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:23.514706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f6a6b-7b85-4c1c-9ca5-78f1e9c2372c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:23.515304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f6a6b-7b85-4c1c-9ca5-78f1e9c2372c service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:23.515579 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f6a6b-7b85-4c1c-9ca5-78f1e9c2372c service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:23.515846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f6a6b-7b85-4c1c-9ca5-78f1e9c2372c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:23.517608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c38f6a6b-7b85-4c1c-9ca5-78f1e9c2372c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:23.624280 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f6a6b-7b85-4c1c-9ca5-78f1e9c2372c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:23.625784 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f6a6b-7b85-4c1c-9ca5-78f1e9c2372c service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:23.645006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 721/1449] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:25.751762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ee7e6-389f-4770-b617-d5d0787adba6 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:25.752621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ee7e6-389f-4770-b617-d5d0787adba6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:25.752621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ee7e6-389f-4770-b617-d5d0787adba6 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:25.753917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:25.759680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38270-11ec-4227-a463-8a92151f121b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:25.760118 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38270-11ec-4227-a463-8a92151f121b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:25.760224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38270-11ec-4227-a463-8a92151f121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:25.831421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-81e38270-11ec-4227-a463-8a92151f121b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:25.832667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-81e38270-11ec-4227-a463-8a92151f121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:25.852668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:25.928074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:26.215130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ee7e6-389f-4770-b617-d5d0787adba6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:26.216735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b00ee7e6-389f-4770-b617-d5d0787adba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:26.331323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ee7e6-389f-4770-b617-d5d0787adba6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:26.351264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:26.406673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de9fa3-f2a3-4fff-820a-6b60067c0d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:26.407224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de9fa3-f2a3-4fff-820a-6b60067c0d80 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:26.407505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de9fa3-f2a3-4fff-820a-6b60067c0d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:26.408446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de9fa3-f2a3-4fff-820a-6b60067c0d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:26.410100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-35de9fa3-f2a3-4fff-820a-6b60067c0d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:26.520943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de9fa3-f2a3-4fff-820a-6b60067c0d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:26.522403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de9fa3-f2a3-4fff-820a-6b60067c0d80 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:26.541408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 730/1452] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:28.509745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c89ae-a4a7-460a-9cd7-c3f3d6635f5e None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:28.510296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c89ae-a4a7-460a-9cd7-c3f3d6635f5e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:28.510296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c89ae-a4a7-460a-9cd7-c3f3d6635f5e None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:28.511643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:28.516908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ea7e-906c-43e0-b99e-a712b5776c67 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:28.517382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ea7e-906c-43e0-b99e-a712b5776c67 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:28.517482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170ea7e-906c-43e0-b99e-a712b5776c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:28.583869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:28.586687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a170ea7e-906c-43e0-b99e-a712b5776c67 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:28.588504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a170ea7e-906c-43e0-b99e-a712b5776c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:28.687490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:28.970885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c89ae-a4a7-460a-9cd7-c3f3d6635f5e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:28.972428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0f6c89ae-a4a7-460a-9cd7-c3f3d6635f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:29.206535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c89ae-a4a7-460a-9cd7-c3f3d6635f5e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:29.255514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:29.280094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a394d-b61b-4ae9-a916-8b3958a28a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:29.280712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a394d-b61b-4ae9-a916-8b3958a28a0f service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:29.280986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a394d-b61b-4ae9-a916-8b3958a28a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:29.281250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a394d-b61b-4ae9-a916-8b3958a28a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:29.283444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bf7a394d-b61b-4ae9-a916-8b3958a28a0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:29.393082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a394d-b61b-4ae9-a916-8b3958a28a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:29.394535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a394d-b61b-4ae9-a916-8b3958a28a0f service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:29.412463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 724/1455] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:11:31.419575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604778-f73d-4c50-b65e-fa542914865a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:31.420099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604778-f73d-4c50-b65e-fa542914865a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:31.420099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604778-f73d-4c50-b65e-fa542914865a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:31.421415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:11:31.427550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecd48-635f-42ef-91eb-4010fb5b4377 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:31.427990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecd48-635f-42ef-91eb-4010fb5b4377 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:31.428127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecd48-635f-42ef-91eb-4010fb5b4377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:31.497823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2b6ecd48-635f-42ef-91eb-4010fb5b4377 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:31.499039 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2b6ecd48-635f-42ef-91eb-4010fb5b4377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:31.531654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:31.606910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:31.901852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02604778-f73d-4c50-b65e-fa542914865a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:31.903388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-02604778-f73d-4c50-b65e-fa542914865a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:32.007170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02604778-f73d-4c50-b65e-fa542914865a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:32.035972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:32.082257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a442f76-4448-4cb0-b854-c3b46f63093e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:32.082896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a442f76-4448-4cb0-b854-c3b46f63093e service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:32.083175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a442f76-4448-4cb0-b854-c3b46f63093e service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:32.083450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a442f76-4448-4cb0-b854-c3b46f63093e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:32.085109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6a442f76-4448-4cb0-b854-c3b46f63093e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:32.196319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a442f76-4448-4cb0-b854-c3b46f63093e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:32.197799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a442f76-4448-4cb0-b854-c3b46f63093e service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:32.215925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 733/1458] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:34.202062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664805f6-8b05-46c7-b85e-90419bc4a520 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:34.202608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664805f6-8b05-46c7-b85e-90419bc4a520 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:34.202608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664805f6-8b05-46c7-b85e-90419bc4a520 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:34.204052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:34.210025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01450495-8702-4e02-a4f8-faf127ccd8fd None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:34.210539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01450495-8702-4e02-a4f8-faf127ccd8fd None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:34.210775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01450495-8702-4e02-a4f8-faf127ccd8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:34.284915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-01450495-8702-4e02-a4f8-faf127ccd8fd None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:34.286147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-01450495-8702-4e02-a4f8-faf127ccd8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:34.295405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:34.394412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:34.679206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664805f6-8b05-46c7-b85e-90419bc4a520 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:34.680776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-664805f6-8b05-46c7-b85e-90419bc4a520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:34.803037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664805f6-8b05-46c7-b85e-90419bc4a520 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:34.865411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:34.886323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993eba79-4a8e-4a9f-be61-5059d5e61eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:34.886934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eba79-4a8e-4a9f-be61-5059d5e61eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:34.887216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eba79-4a8e-4a9f-be61-5059d5e61eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:34.887472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993eba79-4a8e-4a9f-be61-5059d5e61eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:34.889289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-993eba79-4a8e-4a9f-be61-5059d5e61eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:35.000897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eba79-4a8e-4a9f-be61-5059d5e61eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:35.003464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993eba79-4a8e-4a9f-be61-5059d5e61eb4 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:35.022260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 727/1461] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:37.024583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6496-67df-4b14-a175-50b552f915bb None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:37.025166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6496-67df-4b14-a175-50b552f915bb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:37.025166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6496-67df-4b14-a175-50b552f915bb None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:37.026488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:37.031752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381cd3f-98bd-4c10-ba23-902e8a1b12b5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:37.032207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381cd3f-98bd-4c10-ba23-902e8a1b12b5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:37.032389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381cd3f-98bd-4c10-ba23-902e8a1b12b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:37.099170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f381cd3f-98bd-4c10-ba23-902e8a1b12b5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:37.100403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f381cd3f-98bd-4c10-ba23-902e8a1b12b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:37.138831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:37.198616 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:37.485205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e6496-67df-4b14-a175-50b552f915bb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:37.487432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4a8e6496-67df-4b14-a175-50b552f915bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:37.589219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e6496-67df-4b14-a175-50b552f915bb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:37.626293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:37.662271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392ab4d-c0f8-4247-86d6-71b069360e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:37.662872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392ab4d-c0f8-4247-86d6-71b069360e3c service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:37.663156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392ab4d-c0f8-4247-86d6-71b069360e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:37.663427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392ab4d-c0f8-4247-86d6-71b069360e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:37.665044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c392ab4d-c0f8-4247-86d6-71b069360e3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:37.772332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392ab4d-c0f8-4247-86d6-71b069360e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:37.773831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392ab4d-c0f8-4247-86d6-71b069360e3c service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:37.791950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 736/1464] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:39.752632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c6f3-a2a6-4eca-b148-e0a2e5946563 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:39.753251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c6f3-a2a6-4eca-b148-e0a2e5946563 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:39.753251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c6f3-a2a6-4eca-b148-e0a2e5946563 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:39.754503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:39.760603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595343-6419-4dcb-afeb-4b69f74d256d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:39.760603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595343-6419-4dcb-afeb-4b69f74d256d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:39.761094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb595343-6419-4dcb-afeb-4b69f74d256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:39.828816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-eb595343-6419-4dcb-afeb-4b69f74d256d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:39.830769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-eb595343-6419-4dcb-afeb-4b69f74d256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:39.848887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:39.931088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:40.218271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c6f3-a2a6-4eca-b148-e0a2e5946563 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:40.219873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3a91c6f3-a2a6-4eca-b148-e0a2e5946563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:40.326016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c6f3-a2a6-4eca-b148-e0a2e5946563 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:40.393887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:40.400885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0b11b-ca53-458d-bc39-fa13048274b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:40.401489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b11b-ca53-458d-bc39-fa13048274b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:40.401769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b11b-ca53-458d-bc39-fa13048274b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:40.402043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0b11b-ca53-458d-bc39-fa13048274b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:40.403634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-46e0b11b-ca53-458d-bc39-fa13048274b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:40.514015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0b11b-ca53-458d-bc39-fa13048274b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:40.515446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0b11b-ca53-458d-bc39-fa13048274b0 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:40.533905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 730/1467] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:42.529830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b9f51-832e-4ada-94a4-fbffe8b547f6 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:42.530404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b9f51-832e-4ada-94a4-fbffe8b547f6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:42.530404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b9f51-832e-4ada-94a4-fbffe8b547f6 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:42.531722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:11:42.537543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c702-e46e-414d-9e17-f1ba4c8bcb64 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:42.538008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c702-e46e-414d-9e17-f1ba4c8bcb64 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:42.538133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966c702-e46e-414d-9e17-f1ba4c8bcb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:42.603704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7966c702-e46e-414d-9e17-f1ba4c8bcb64 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:11:42.604951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7966c702-e46e-414d-9e17-f1ba4c8bcb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:42.653977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:42.704309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:11:42.983398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b9f51-832e-4ada-94a4-fbffe8b547f6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:11:42.984987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ec8b9f51-832e-4ada-94a4-fbffe8b547f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:43.091089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b9f51-832e-4ada-94a4-fbffe8b547f6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:11:43.139679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:43.166402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e37030-94a0-4793-9e81-1b43e633cf72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:11:43.167007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37030-94a0-4793-9e81-1b43e633cf72 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:11:43.167289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37030-94a0-4793-9e81-1b43e633cf72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:11:43.167827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37030-94a0-4793-9e81-1b43e633cf72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:11:43.169466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-72e37030-94a0-4793-9e81-1b43e633cf72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:11:43.280485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e37030-94a0-4793-9e81-1b43e633cf72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:11:43.281898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e37030-94a0-4793-9e81-1b43e633cf72 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:11:43.301388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 739/1470] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:11:52.702137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=22 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:11:53.298424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=66 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.026659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e7322-bcaf-44e0-9329-d5a2cccaeb78 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.027251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e7322-bcaf-44e0-9329-d5a2cccaeb78 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.027251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e7322-bcaf-44e0-9329-d5a2cccaeb78 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.028684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 732/1471] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:03.034681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5038964-a5a9-48b0-959d-61fda653ef2d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.035131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5038964-a5a9-48b0-959d-61fda653ef2d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.035239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5038964-a5a9-48b0-959d-61fda653ef2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.102258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c5038964-a5a9-48b0-959d-61fda653ef2d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:03.103487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c5038964-a5a9-48b0-959d-61fda653ef2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:03.203639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 740/1472] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:03.507936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c88d9-6606-4e25-b69c-7fd06ab70949 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.508201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c88d9-6606-4e25-b69c-7fd06ab70949 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.508468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c88d9-6606-4e25-b69c-7fd06ab70949 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.509852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 733/1473] 10.222.0.95 () {54 vars in 967 bytes} [Wed Aug 31 19:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:03.514638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8d8d-531f-4b00-a4b2-928c897fb294 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.514921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8d8d-531f-4b00-a4b2-928c897fb294 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.515188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8d8d-531f-4b00-a4b2-928c897fb294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.592432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c7ce8d8d-531f-4b00-a4b2-928c897fb294 None None] MFA Rules not processed for user `8b6c90a2b95c498494a73b43e7138905`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:03.593680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c7ce8d8d-531f-4b00-a4b2-928c897fb294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:03.614683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.670740 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 741/1474] 10.222.0.95 () {58 vars in 1064 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:03.678979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c88d9-6606-4e25-b69c-7fd06ab70949 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.680504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f75c88d9-6606-4e25-b69c-7fd06ab70949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:03.784347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c88d9-6606-4e25-b69c-7fd06ab70949 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.857479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b115326-868b-473c-a21e-0dc714836655 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6c90a2b95c498494a73b43e7138905', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.858091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b115326-868b-473c-a21e-0dc714836655 service glance] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.858370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b115326-868b-473c-a21e-0dc714836655 service glance] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.858641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b115326-868b-473c-a21e-0dc714836655 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.860811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5b115326-868b-473c-a21e-0dc714836655 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:03.914615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.973760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b115326-868b-473c-a21e-0dc714836655 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.975207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b115326-868b-473c-a21e-0dc714836655 service glance] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.994286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 734/1475] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:06.031770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94529c-a9ca-4fc1-b6bd-316966629156 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.032945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94529c-a9ca-4fc1-b6bd-316966629156 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.032945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94529c-a9ca-4fc1-b6bd-316966629156 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.034577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 742/1476] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:06.039926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edef4d-83e1-4695-a871-b61aeba6b7c2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.040342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edef4d-83e1-4695-a871-b61aeba6b7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.040465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edef4d-83e1-4695-a871-b61aeba6b7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.108346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e4edef4d-83e1-4695-a871-b61aeba6b7c2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:06.109609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e4edef4d-83e1-4695-a871-b61aeba6b7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:06.210692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 735/1477] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:06.487554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94529c-a9ca-4fc1-b6bd-316966629156 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.489238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3a94529c-a9ca-4fc1-b6bd-316966629156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:06.614741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94529c-a9ca-4fc1-b6bd-316966629156 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.663923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:06.709681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee814c89-acd1-421a-9777-3403e52b637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.710284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814c89-acd1-421a-9777-3403e52b637a admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.710557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814c89-acd1-421a-9777-3403e52b637a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.710822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814c89-acd1-421a-9777-3403e52b637a admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.711458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee814c89-acd1-421a-9777-3403e52b637a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:06.713279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING py.warnings [None req-ee814c89-acd1-421a-9777-3403e52b637a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:12:06.713279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: warnings.warn(msg) Aug 31 19:12:06.713279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: Aug 31 19:12:06.714401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee814c89-acd1-421a-9777-3403e52b637a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.733092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 743/1478] 10.222.0.95 () {56 vars in 1108 bytes} [Wed Aug 31 19:12:06 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:08.649211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3589-b699-4bf8-b6a8-5f9d6e1be26a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:08.649854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3589-b699-4bf8-b6a8-5f9d6e1be26a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:08.649854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3589-b699-4bf8-b6a8-5f9d6e1be26a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:08.651172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 736/1479] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:08.656311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3211-5bf1-4ceb-96fd-ddff910a55c7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:08.656780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3211-5bf1-4ceb-96fd-ddff910a55c7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:08.656893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb3211-5bf1-4ceb-96fd-ddff910a55c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:08.723929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-72bb3211-5bf1-4ceb-96fd-ddff910a55c7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:08.725181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-72bb3211-5bf1-4ceb-96fd-ddff910a55c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:08.821822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 744/1480] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:09.117491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3589-b699-4bf8-b6a8-5f9d6e1be26a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:09.119015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-673e3589-b699-4bf8-b6a8-5f9d6e1be26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:09.129598 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:09.288474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3589-b699-4bf8-b6a8-5f9d6e1be26a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:09.376948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:09.410681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545839e2-942d-462e-b4a4-6b154999ebc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:09.411281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545839e2-942d-462e-b4a4-6b154999ebc6 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:09.411583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545839e2-942d-462e-b4a4-6b154999ebc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:09.411848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545839e2-942d-462e-b4a4-6b154999ebc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:09.413475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-545839e2-942d-462e-b4a4-6b154999ebc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:09.506438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545839e2-942d-462e-b4a4-6b154999ebc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:09.508027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545839e2-942d-462e-b4a4-6b154999ebc6 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:09.526127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 737/1481] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:11.658166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d341a-e610-4cd3-9bcf-4ce48cddbbc0 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:11.658808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d341a-e610-4cd3-9bcf-4ce48cddbbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:11.658808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d341a-e610-4cd3-9bcf-4ce48cddbbc0 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:11.660066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 745/1482] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:11.665439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec4616-cdac-4dc9-ae73-e43375dcdf81 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:11.665856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec4616-cdac-4dc9-ae73-e43375dcdf81 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:11.665992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec4616-cdac-4dc9-ae73-e43375dcdf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:11.732120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cdec4616-cdac-4dc9-ae73-e43375dcdf81 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:11.733489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cdec4616-cdac-4dc9-ae73-e43375dcdf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:11.781178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:11.831989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 738/1483] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:12.122507 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d341a-e610-4cd3-9bcf-4ce48cddbbc0 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:12.124156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0f7d341a-e610-4cd3-9bcf-4ce48cddbbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:12.150027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:12.240070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d341a-e610-4cd3-9bcf-4ce48cddbbc0 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:12.318423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec54c87-1e8b-4ffb-88c8-5b3d7fb53841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:12.319036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec54c87-1e8b-4ffb-88c8-5b3d7fb53841 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:12.319319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec54c87-1e8b-4ffb-88c8-5b3d7fb53841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:12.319592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec54c87-1e8b-4ffb-88c8-5b3d7fb53841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:12.321241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-eec54c87-1e8b-4ffb-88c8-5b3d7fb53841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:12.395569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:12.431517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec54c87-1e8b-4ffb-88c8-5b3d7fb53841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:12.433084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec54c87-1e8b-4ffb-88c8-5b3d7fb53841 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:12.451622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 746/1484] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:14.476336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8dd99-64cb-45a7-92c0-b2e751d95183 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.476928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8dd99-64cb-45a7-92c0-b2e751d95183 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.476928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8dd99-64cb-45a7-92c0-b2e751d95183 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.478262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 739/1485] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:14.483499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8320d-c840-4328-a55c-34cecdb01314 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.483903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8320d-c840-4328-a55c-34cecdb01314 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.484056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8320d-c840-4328-a55c-34cecdb01314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.552214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-77f8320d-c840-4328-a55c-34cecdb01314 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:14.553460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-77f8320d-c840-4328-a55c-34cecdb01314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:14.651192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 747/1486] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:14.943025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8dd99-64cb-45a7-92c0-b2e751d95183 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:14.944583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-31d8dd99-64cb-45a7-92c0-b2e751d95183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:15.051050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8dd99-64cb-45a7-92c0-b2e751d95183 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:15.123970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:15.126381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3296e57-7df8-43f5-8bdf-d93569dc7b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:15.126989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296e57-7df8-43f5-8bdf-d93569dc7b1b service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:15.127344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296e57-7df8-43f5-8bdf-d93569dc7b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:15.127843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3296e57-7df8-43f5-8bdf-d93569dc7b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:15.129945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a3296e57-7df8-43f5-8bdf-d93569dc7b1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:15.242232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3296e57-7df8-43f5-8bdf-d93569dc7b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:15.243702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3296e57-7df8-43f5-8bdf-d93569dc7b1b service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:15.262512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 740/1487] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:17.293644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48df0e-9ebf-40fa-a4ae-d891820e1fa1 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:17.294209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48df0e-9ebf-40fa-a4ae-d891820e1fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:17.294209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48df0e-9ebf-40fa-a4ae-d891820e1fa1 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:17.295541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 748/1488] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:17.300822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174b45-4037-40dc-9f2f-202a3c7bdccf None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:17.301291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174b45-4037-40dc-9f2f-202a3c7bdccf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:17.301396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e174b45-4037-40dc-9f2f-202a3c7bdccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:17.366877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8e174b45-4037-40dc-9f2f-202a3c7bdccf None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:17.368074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8e174b45-4037-40dc-9f2f-202a3c7bdccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:17.421162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:17.465980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 741/1489] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:17.761183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48df0e-9ebf-40fa-a4ae-d891820e1fa1 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:17.762772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cc48df0e-9ebf-40fa-a4ae-d891820e1fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:17.802430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:17.871886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48df0e-9ebf-40fa-a4ae-d891820e1fa1 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:17.951910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55038814-e920-4ade-8c06-db8b401e4abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:17.952520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55038814-e920-4ade-8c06-db8b401e4abf service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:17.952845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55038814-e920-4ade-8c06-db8b401e4abf service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:17.953096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55038814-e920-4ade-8c06-db8b401e4abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:17.955729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-55038814-e920-4ade-8c06-db8b401e4abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:18.036239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:18.065526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55038814-e920-4ade-8c06-db8b401e4abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:18.066996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55038814-e920-4ade-8c06-db8b401e4abf service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:18.086314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 749/1490] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:20.320353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20ccdc-709c-4427-851c-6f0f7e150d09 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:20.320962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20ccdc-709c-4427-851c-6f0f7e150d09 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:20.320962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20ccdc-709c-4427-851c-6f0f7e150d09 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:20.322307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 742/1491] 10.222.0.95 () {54 vars in 927 bytes} [Wed Aug 31 19:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:20.327513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3ad24-4367-4348-8838-b75c925d0e86 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:20.327954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3ad24-4367-4348-8838-b75c925d0e86 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:20.328147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3ad24-4367-4348-8838-b75c925d0e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:20.395516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-cba3ad24-4367-4348-8838-b75c925d0e86 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:20.396668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cba3ad24-4367-4348-8838-b75c925d0e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:20.493169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 750/1492] 10.222.0.95 () {58 vars in 1024 bytes} [Wed Aug 31 19:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:20.788390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20ccdc-709c-4427-851c-6f0f7e150d09 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:20.789972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9c20ccdc-709c-4427-851c-6f0f7e150d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:20.893864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20ccdc-709c-4427-851c-6f0f7e150d09 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:20.967379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb70352-66e2-4032-a708-63dbe1f675e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:20.967985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb70352-66e2-4032-a708-63dbe1f675e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:20.968263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb70352-66e2-4032-a708-63dbe1f675e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:20.968531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb70352-66e2-4032-a708-63dbe1f675e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:20.970473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5fb70352-66e2-4032-a708-63dbe1f675e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:20.979611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:21.082343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb70352-66e2-4032-a708-63dbe1f675e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:21.083990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb70352-66e2-4032-a708-63dbe1f675e2 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:21.103183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 743/1493] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:30.491221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=82 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:31.101797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=50 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:36.341345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c683f8b-81b8-4e43-a957-2a604f0bf446 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.341945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c683f8b-81b8-4e43-a957-2a604f0bf446 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.341945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c683f8b-81b8-4e43-a957-2a604f0bf446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.420553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6c683f8b-81b8-4e43-a957-2a604f0bf446 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:36.422190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6c683f8b-81b8-4e43-a957-2a604f0bf446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:36.523487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 751/1494] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:36.532442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2341-6212-4a49-a2fe-90d42865caf3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.533021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2341-6212-4a49-a2fe-90d42865caf3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.533021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a2341-6212-4a49-a2fe-90d42865caf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.610432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-423a2341-6212-4a49-a2fe-90d42865caf3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:36.611697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-423a2341-6212-4a49-a2fe-90d42865caf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:36.706313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 744/1495] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:36.710926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c683f8b-81b8-4e43-a957-2a604f0bf446 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:36.712136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6c683f8b-81b8-4e43-a957-2a604f0bf446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:36.723739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c683f8b-81b8-4e43-a957-2a604f0bf446 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:36.784875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:36.819666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:36.820261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.820537 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.820834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.821472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:36.821779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:36.822796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING py.warnings [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:13:36.822796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: warnings.warn(msg) Aug 31 19:13:36.822796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: Aug 31 19:13:36.823683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:36.835924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 752/1496] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:36.844135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a2341-6212-4a49-a2fe-90d42865caf3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:36.853312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a2341-6212-4a49-a2fe-90d42865caf3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:36.914839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:36.949841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:36.950447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.950724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.950992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.951626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:36.954031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:13:36.954031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Aug 31 19:13:36.954031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: Aug 31 19:13:36.955221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.010431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 745/1497] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:36 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:37.014627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.017257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:37.051690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7776a43-fe3c-4b75-ae5d-c0fb66c81a6b admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.149703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.150308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.150699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.150998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.151698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.152979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING py.warnings [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:13:37.152979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: warnings.warn(msg) Aug 31 19:13:37.152979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: Aug 31 19:13:37.154207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.196941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 753/1498] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:37 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:37.201142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.202697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:37.264418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59452db1-5d68-4b41-86fc-1d1d0fae805e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.347133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:37.358012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.358610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.358910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.359179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.359815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.360838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:13:37.360838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Aug 31 19:13:37.360838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: Aug 31 19:13:37.361720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.373870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 746/1499] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:37.378093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.379597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:37.391727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a103d-dc81-4ca3-8ab8-ee0e8f92a72c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.417155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:37.489734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052d5c5-2505-4062-8649-babda7f33f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.490344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052d5c5-2505-4062-8649-babda7f33f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.490621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052d5c5-2505-4062-8649-babda7f33f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.490893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052d5c5-2505-4062-8649-babda7f33f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.491541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052d5c5-2505-4062-8649-babda7f33f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.492604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING py.warnings [None req-d052d5c5-2505-4062-8649-babda7f33f70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:13:37.492604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: warnings.warn(msg) Aug 31 19:13:37.492604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: Aug 31 19:13:37.493542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052d5c5-2505-4062-8649-babda7f33f70 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.506424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 754/1500] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:37.510710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.512289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:37.548181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c3524-e9f4-4820-91d3-d842f0363162 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.642985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd073d-f6d8-4b37-a73f-28348af82421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.643639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd073d-f6d8-4b37-a73f-28348af82421 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.643918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd073d-f6d8-4b37-a73f-28348af82421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.644186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd073d-f6d8-4b37-a73f-28348af82421 admin admin] PATH_INFO: `/v3/projects/3af6fa2f369a4d7fae0aa40f7324f4f7/users/379cdcf56a5540138569d14a7c179703/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.666623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd073d-f6d8-4b37-a73f-28348af82421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af6fa2f369a4d7fae0aa40f7324f4f7, user_id=379cdcf56a5540138569d14a7c179703, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.667936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-c2cd073d-f6d8-4b37-a73f-28348af82421 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:13:37.667936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Aug 31 19:13:37.667936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: Aug 31 19:13:37.669922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd073d-f6d8-4b37-a73f-28348af82421 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.694259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 747/1501] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:37 2022] PUT /identity/v3/projects/3af6fa2f369a4d7fae0aa40f7324f4f7/users/379cdcf56a5540138569d14a7c179703/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:37.706958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525748f7-39f1-4308-b401-f157dcc838e2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.707252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525748f7-39f1-4308-b401-f157dcc838e2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.707532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525748f7-39f1-4308-b401-f157dcc838e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.769831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-525748f7-39f1-4308-b401-f157dcc838e2 None None] MFA Rules not processed for user `379cdcf56a5540138569d14a7c179703`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:37.771060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-525748f7-39f1-4308-b401-f157dcc838e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:37.847844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 755/1502] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:13:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:37.861117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ab3c-5986-4f5b-b315-842e5a796e48 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.861412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ab3c-5986-4f5b-b315-842e5a796e48 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.861688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1ab3c-5986-4f5b-b315-842e5a796e48 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.863041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 748/1503] 10.222.0.95 () {54 vars in 964 bytes} [Wed Aug 31 19:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:13:37.867948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fea536-af7d-4025-b45a-9e2aa0014b9b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.868238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fea536-af7d-4025-b45a-9e2aa0014b9b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.868515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fea536-af7d-4025-b45a-9e2aa0014b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.900882 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:37.945916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-29fea536-af7d-4025-b45a-9e2aa0014b9b None None] MFA Rules not processed for user `115298313c1243d282d09a8e8005002c`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:37.947119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-29fea536-af7d-4025-b45a-9e2aa0014b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.023996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 756/1504] 10.222.0.95 () {58 vars in 1061 bytes} [Wed Aug 31 19:13:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:38.033703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ab3c-5986-4f5b-b315-842e5a796e48 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:38.035210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-54c1ab3c-5986-4f5b-b315-842e5a796e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.088231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:38.145646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1ab3c-5986-4f5b-b315-842e5a796e48 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:38.220217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:38.220829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.221112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.221383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.222973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.326181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:38.328939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:38.332523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:38.350276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 749/1505] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:38.399912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520f4e3-8897-46b4-9d59-65c34568a09d None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.400439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520f4e3-8897-46b4-9d59-65c34568a09d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.400439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520f4e3-8897-46b4-9d59-65c34568a09d None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.401867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 757/1506] 10.222.0.95 () {52 vars in 830 bytes} [Wed Aug 31 19:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:13:38.406805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:38.408056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.514230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2149c-6b7c-49af-9bf3-686effe84f80 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:38.608456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:38.609940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.610037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.610254 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] PATH_INFO: `/v3/users/379cdcf56a5540138569d14a7c179703` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.622350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=379cdcf56a5540138569d14a7c179703)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:38.623367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:13:38.623367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Aug 31 19:13:38.623367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: Aug 31 19:13:38.625022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:38.634174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:38.692355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379cdcf56a5540138569d14a7c179703'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.715018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379cdcf56a5540138569d14a7c179703'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.715474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379cdcf56a5540138569d14a7c179703'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.715995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379cdcf56a5540138569d14a7c179703'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.725756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379cdcf56a5540138569d14a7c179703'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.745760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 750/1507] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:38 2022] DELETE /identity/v3/users/379cdcf56a5540138569d14a7c179703 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:38.750798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe481a-6205-4599-8b1b-89191ae296ed None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.751065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe481a-6205-4599-8b1b-89191ae296ed None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.751345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe481a-6205-4599-8b1b-89191ae296ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.807980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6cfe481a-6205-4599-8b1b-89191ae296ed None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:38.809222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6cfe481a-6205-4599-8b1b-89191ae296ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.886330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:38.899475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 758/1508] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:38.909165 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:38.910425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:39.010772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d832d85-86e0-4b1a-8ab0-d1ec6d02eb2b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:39.065995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:39.087483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3b835-d331-4ac0-bb52-008cb0ad688e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:39.088125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3b835-d331-4ac0-bb52-008cb0ad688e service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:39.088943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3b835-d331-4ac0-bb52-008cb0ad688e service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:39.089209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3b835-d331-4ac0-bb52-008cb0ad688e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:39.090992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b5b3b835-d331-4ac0-bb52-008cb0ad688e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:39.206829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3b835-d331-4ac0-bb52-008cb0ad688e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:39.208187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3b835-d331-4ac0-bb52-008cb0ad688e service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:39.226130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 751/1509] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:39.558711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe481a-6205-4599-8b1b-89191ae296ed None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:39.560034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6cfe481a-6205-4599-8b1b-89191ae296ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:39.572610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe481a-6205-4599-8b1b-89191ae296ed None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:39.673969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:39.674591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:39.674866 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:39.675131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] PATH_INFO: `/v3/projects/3af6fa2f369a4d7fae0aa40f7324f4f7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:39.685289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3af6fa2f369a4d7fae0aa40f7324f4f7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:39.686318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING py.warnings [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:13:39.686318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: warnings.warn(msg) Aug 31 19:13:39.686318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: Aug 31 19:13:39.687703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:39.748751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3af6fa2f369a4d7fae0aa40f7324f4f7', 'request_id': 'req-3b50eccc-b239-47a8-9024-e339acbb9a9d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.762891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3af6fa2f369a4d7fae0aa40f7324f4f7', 'request_id': 'req-3b50eccc-b239-47a8-9024-e339acbb9a9d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.769163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3af6fa2f369a4d7fae0aa40f7324f4f7', 'request_id': 'req-3b50eccc-b239-47a8-9024-e339acbb9a9d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.795660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] The token cache is being invalidate because project 3af6fa2f369a4d7fae0aa40f7324f4f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:39.796012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.796981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b50eccc-b239-47a8-9024-e339acbb9a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.798408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 759/1510] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:39 2022] DELETE /identity/v3/projects/3af6fa2f369a4d7fae0aa40f7324f4f7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:49.223793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=62 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:49.793642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:16.036088 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d568311-64f7-4f1b-ac2d-93a25656e12f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:16.036773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d568311-64f7-4f1b-ac2d-93a25656e12f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:16.036773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d568311-64f7-4f1b-ac2d-93a25656e12f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:16.038044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:15:16.044823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c4565-93a9-431b-88c3-120ab7ec7d92 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:16.045408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c4565-93a9-431b-88c3-120ab7ec7d92 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:16.045408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c4565-93a9-431b-88c3-120ab7ec7d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:16.130197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-893c4565-93a9-431b-88c3-120ab7ec7d92 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:16.131389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-893c4565-93a9-431b-88c3-120ab7ec7d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:16.241094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:15:16.402049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d568311-64f7-4f1b-ac2d-93a25656e12f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:16.404412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0d568311-64f7-4f1b-ac2d-93a25656e12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:16.534120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d568311-64f7-4f1b-ac2d-93a25656e12f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:16.540672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca32ff-cda0-4d33-a207-86265dc23bd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:16.541830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca32ff-cda0-4d33-a207-86265dc23bd1 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:16.542223 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca32ff-cda0-4d33-a207-86265dc23bd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:16.542592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca32ff-cda0-4d33-a207-86265dc23bd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:16.551501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca32ff-cda0-4d33-a207-86265dc23bd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:16.553133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca32ff-cda0-4d33-a207-86265dc23bd1 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:16.562697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 753/1513] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:17.433404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df289df-a278-45db-a3b8-69140190e2f7 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:17.433976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df289df-a278-45db-a3b8-69140190e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:17.433976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df289df-a278-45db-a3b8-69140190e2f7 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:17.435234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:17.440265 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726392-7eeb-4b1c-b422-ce03b8ef164f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:17.440797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726392-7eeb-4b1c-b422-ce03b8ef164f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:17.440907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b726392-7eeb-4b1c-b422-ce03b8ef164f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:17.465198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2b726392-7eeb-4b1c-b422-ce03b8ef164f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:17.466419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2b726392-7eeb-4b1c-b422-ce03b8ef164f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:17.474828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:17.633262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df289df-a278-45db-a3b8-69140190e2f7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:17.648486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df289df-a278-45db-a3b8-69140190e2f7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:17.654084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692ce91-bd8c-43be-9bf4-65e88f7c1cac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:17.654688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ce91-bd8c-43be-9bf4-65e88f7c1cac service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:17.654974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ce91-bd8c-43be-9bf4-65e88f7c1cac service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:17.655261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ce91-bd8c-43be-9bf4-65e88f7c1cac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:17.664278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692ce91-bd8c-43be-9bf4-65e88f7c1cac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:17.665941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692ce91-bd8c-43be-9bf4-65e88f7c1cac service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:17.675930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 762/1516] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:20.778314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de143cee-cc43-4ba0-826a-e91a0479af31 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:20.778925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de143cee-cc43-4ba0-826a-e91a0479af31 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:20.778925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de143cee-cc43-4ba0-826a-e91a0479af31 None None] PATH_INFO: `/v3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:20.780175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 755/1517] 10.222.0.13 () {54 vars in 925 bytes} [Wed Aug 31 19:15:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:15:20.790159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd219e-79ee-4054-a632-6f3559d3e961 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:20.790699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd219e-79ee-4054-a632-6f3559d3e961 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:20.791114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd219e-79ee-4054-a632-6f3559d3e961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:20.879993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c2cd219e-79ee-4054-a632-6f3559d3e961 None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:20.882428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c2cd219e-79ee-4054-a632-6f3559d3e961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:20.901479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:20.982765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 763/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Aug 31 19:15:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:27.459973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=56 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:28.917194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d5b04-8b38-4e38-9f82-8487fd2cb636 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:28.917873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d5b04-8b38-4e38-9f82-8487fd2cb636 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:28.917873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d5b04-8b38-4e38-9f82-8487fd2cb636 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:28.919092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 756/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:28.925093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b993d-5f89-4343-bb4b-f894a305b189 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:28.925540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b993d-5f89-4343-bb4b-f894a305b189 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:28.925735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b993d-5f89-4343-bb4b-f894a305b189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:28.947543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c03b993d-5f89-4343-bb4b-f894a305b189 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:28.948830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c03b993d-5f89-4343-bb4b-f894a305b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:28.956971 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 764/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:29.119269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d5b04-8b38-4e38-9f82-8487fd2cb636 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:29.136459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d5b04-8b38-4e38-9f82-8487fd2cb636 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:29.142967 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09dd75-ad6a-46ae-a133-485b8f0c418e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:29.143531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09dd75-ad6a-46ae-a133-485b8f0c418e service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:29.143742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09dd75-ad6a-46ae-a133-485b8f0c418e service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:29.144010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09dd75-ad6a-46ae-a133-485b8f0c418e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:29.153152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09dd75-ad6a-46ae-a133-485b8f0c418e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:29.154965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09dd75-ad6a-46ae-a133-485b8f0c418e service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:29.165504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 757/1521] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:30.002532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48356c-95de-4e01-9d28-bfa3181eab07 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.003109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48356c-95de-4e01-9d28-bfa3181eab07 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.003109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48356c-95de-4e01-9d28-bfa3181eab07 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.004589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:30.009494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650693c-c026-4374-9896-2fae113572c1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.009962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650693c-c026-4374-9896-2fae113572c1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.010058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650693c-c026-4374-9896-2fae113572c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.028971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4650693c-c026-4374-9896-2fae113572c1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:30.030168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4650693c-c026-4374-9896-2fae113572c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:30.038913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:30.198431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48356c-95de-4e01-9d28-bfa3181eab07 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.217175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48356c-95de-4e01-9d28-bfa3181eab07 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.223343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f989b6-def5-4d78-b2f4-4653170fdd9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.223947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f989b6-def5-4d78-b2f4-4653170fdd9f service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.224229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f989b6-def5-4d78-b2f4-4653170fdd9f service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.224503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f989b6-def5-4d78-b2f4-4653170fdd9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.232864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f989b6-def5-4d78-b2f4-4653170fdd9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.234486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f989b6-def5-4d78-b2f4-4653170fdd9f service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.244947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 766/1524] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:32.828803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6a140-e0f3-4d4d-b476-059772e1b245 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.829348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6a140-e0f3-4d4d-b476-059772e1b245 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.829348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6a140-e0f3-4d4d-b476-059772e1b245 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.831182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 759/1525] 10.222.0.95 () {54 vars in 961 bytes} [Wed Aug 31 19:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:32.836283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3edce-10a1-431b-9a16-4918060b6b70 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.836796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3edce-10a1-431b-9a16-4918060b6b70 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.836908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3edce-10a1-431b-9a16-4918060b6b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.976545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fac3edce-10a1-431b-9a16-4918060b6b70 None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:32.977855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fac3edce-10a1-431b-9a16-4918060b6b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:32.987206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 767/1526] 10.222.0.95 () {58 vars in 1058 bytes} [Wed Aug 31 19:15:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:32.994830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6a140-e0f3-4d4d-b476-059772e1b245 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.008311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6a140-e0f3-4d4d-b476-059772e1b245 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.013327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e9350-0d71-4d7b-9d4e-811d25056436 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.013916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9350-0d71-4d7b-9d4e-811d25056436 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.014197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9350-0d71-4d7b-9d4e-811d25056436 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.014473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e9350-0d71-4d7b-9d4e-811d25056436 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.022797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e9350-0d71-4d7b-9d4e-811d25056436 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.024382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e9350-0d71-4d7b-9d4e-811d25056436 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:33.034384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 760/1527] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:41.500303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dd3d-fb4f-4c88-8ab4-c967fd8cc522 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:41.500921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dd3d-fb4f-4c88-8ab4-c967fd8cc522 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:41.500921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dd3d-fb4f-4c88-8ab4-c967fd8cc522 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:41.502244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:41.507799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c0ef-481a-43c8-a9ec-90153d5475e9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:41.508252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c0ef-481a-43c8-a9ec-90153d5475e9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:41.508364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c0ef-481a-43c8-a9ec-90153d5475e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:41.528816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7df7c0ef-481a-43c8-a9ec-90153d5475e9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:41.530015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7df7c0ef-481a-43c8-a9ec-90153d5475e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:41.538089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:41.698493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692dd3d-fb4f-4c88-8ab4-c967fd8cc522 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:41.713238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692dd3d-fb4f-4c88-8ab4-c967fd8cc522 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:41.718787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8f2e2-0c9a-4d83-954e-d2714808a87a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:41.719378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8f2e2-0c9a-4d83-954e-d2714808a87a service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:41.719654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8f2e2-0c9a-4d83-954e-d2714808a87a service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:41.719923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8f2e2-0c9a-4d83-954e-d2714808a87a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:41.728721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8f2e2-0c9a-4d83-954e-d2714808a87a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:41.730316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8f2e2-0c9a-4d83-954e-d2714808a87a service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:41.739611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 769/1530] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:42.569895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabf65b-c055-47ec-9333-544945b05fd4 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabf65b-c055-47ec-9333-544945b05fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.570480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabf65b-c055-47ec-9333-544945b05fd4 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.571789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:42.576628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1368f-f158-4844-a9cb-ef68e5143eb3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.577092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1368f-f158-4844-a9cb-ef68e5143eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.577189 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1368f-f158-4844-a9cb-ef68e5143eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.596686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-83c1368f-f158-4844-a9cb-ef68e5143eb3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:42.597886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-83c1368f-f158-4844-a9cb-ef68e5143eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:42.605500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:42.756374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabf65b-c055-47ec-9333-544945b05fd4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:42.771771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabf65b-c055-47ec-9333-544945b05fd4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:42.777435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c857899-db4d-4fbc-a2f1-a1894a09017e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:42.778034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857899-db4d-4fbc-a2f1-a1894a09017e service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.778314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857899-db4d-4fbc-a2f1-a1894a09017e service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.778585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857899-db4d-4fbc-a2f1-a1894a09017e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.787676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c857899-db4d-4fbc-a2f1-a1894a09017e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.789352 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c857899-db4d-4fbc-a2f1-a1894a09017e service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.799444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 763/1533] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:46.638367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc32f9b-b45e-4cd1-9144-1c46ffca5c80 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:46.638983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc32f9b-b45e-4cd1-9144-1c46ffca5c80 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:46.638983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc32f9b-b45e-4cd1-9144-1c46ffca5c80 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:46.640267 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:46.646172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3762a7-78bf-4b7d-aefa-20150b20f093 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:46.646642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3762a7-78bf-4b7d-aefa-20150b20f093 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:46.646757 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3762a7-78bf-4b7d-aefa-20150b20f093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:46.668186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9a3762a7-78bf-4b7d-aefa-20150b20f093 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:46.669423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9a3762a7-78bf-4b7d-aefa-20150b20f093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:46.677756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:46.974523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc32f9b-b45e-4cd1-9144-1c46ffca5c80 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:46.976264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cdc32f9b-b45e-4cd1-9144-1c46ffca5c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:47.054714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc32f9b-b45e-4cd1-9144-1c46ffca5c80 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:47.058236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33282c2-6f2d-4bea-98a4-aeda6cc3ebac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:47.058832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33282c2-6f2d-4bea-98a4-aeda6cc3ebac service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:47.059106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33282c2-6f2d-4bea-98a4-aeda6cc3ebac service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:47.059372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33282c2-6f2d-4bea-98a4-aeda6cc3ebac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:47.069098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33282c2-6f2d-4bea-98a4-aeda6cc3ebac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:47.070540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33282c2-6f2d-4bea-98a4-aeda6cc3ebac service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:47.079788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 772/1536] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:48.998982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd974-5741-4bd4-bf41-6c9dfd57c6ec None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:48.999569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd974-5741-4bd4-bf41-6c9dfd57c6ec None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:48.999569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdd974-5741-4bd4-bf41-6c9dfd57c6ec None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:49.001082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:49.007116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37493e24-950e-4f5f-bc85-9480da49b29b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:49.007572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37493e24-950e-4f5f-bc85-9480da49b29b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:49.007680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37493e24-950e-4f5f-bc85-9480da49b29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:49.027892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-37493e24-950e-4f5f-bc85-9480da49b29b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:49.029135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-37493e24-950e-4f5f-bc85-9480da49b29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:49.038406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:49.332505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdd974-5741-4bd4-bf41-6c9dfd57c6ec None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:49.343208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdd974-5741-4bd4-bf41-6c9dfd57c6ec None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:49.346891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d8229-9def-443b-b88a-65a55640b542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:49.347555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d8229-9def-443b-b88a-65a55640b542 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:49.347833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d8229-9def-443b-b88a-65a55640b542 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:49.348103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d8229-9def-443b-b88a-65a55640b542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:49.356865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d8229-9def-443b-b88a-65a55640b542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:49.358301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d8229-9def-443b-b88a-65a55640b542 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:49.367525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 766/1539] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:52.884235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c23d-6b19-43f4-8ab1-adbd7eb71501 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:52.884887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c23d-6b19-43f4-8ab1-adbd7eb71501 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:52.884887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c23d-6b19-43f4-8ab1-adbd7eb71501 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:52.886146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 774/1540] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:52.891290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e36ed-cfee-4cb6-ae52-793bcecc505e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:52.891727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e36ed-cfee-4cb6-ae52-793bcecc505e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:52.891834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e36ed-cfee-4cb6-ae52-793bcecc505e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:52.912335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-df0e36ed-cfee-4cb6-ae52-793bcecc505e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:52.913632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-df0e36ed-cfee-4cb6-ae52-793bcecc505e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:52.921670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 767/1541] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:53.480075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40c23d-6b19-43f4-8ab1-adbd7eb71501 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:53.481769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8e40c23d-6b19-43f4-8ab1-adbd7eb71501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:53.485822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=83 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:53.562345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40c23d-6b19-43f4-8ab1-adbd7eb71501 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:53.566517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c4497-c98a-42f6-a794-6b9ad224f57f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:53.567115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c4497-c98a-42f6-a794-6b9ad224f57f service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:53.567398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c4497-c98a-42f6-a794-6b9ad224f57f service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:53.567667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c4497-c98a-42f6-a794-6b9ad224f57f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:53.578383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c4497-c98a-42f6-a794-6b9ad224f57f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:53.579831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c4497-c98a-42f6-a794-6b9ad224f57f service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:53.589462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 775/1542] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:55.576792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbb4cd-e71d-436e-9fb7-bd1c68521d7c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:55.577397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbb4cd-e71d-436e-9fb7-bd1c68521d7c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:55.577397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbb4cd-e71d-436e-9fb7-bd1c68521d7c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:55.578695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 768/1543] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:15:55.583994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf0129-f816-4719-9c44-56b4bf51a002 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:55.584511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf0129-f816-4719-9c44-56b4bf51a002 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:55.584511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf0129-f816-4719-9c44-56b4bf51a002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:55.603947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-93bf0129-f816-4719-9c44-56b4bf51a002 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:55.605178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-93bf0129-f816-4719-9c44-56b4bf51a002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:55.613398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 776/1544] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:56.169225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbb4cd-e71d-436e-9fb7-bd1c68521d7c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:56.170862 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-16dbb4cd-e71d-436e-9fb7-bd1c68521d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:56.184238 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbb4cd-e71d-436e-9fb7-bd1c68521d7c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:56.187680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df53b0-bd09-4102-8d94-8ddffbfb5020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:56.188290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df53b0-bd09-4102-8d94-8ddffbfb5020 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:56.188583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df53b0-bd09-4102-8d94-8ddffbfb5020 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:56.188856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df53b0-bd09-4102-8d94-8ddffbfb5020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:56.198172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df53b0-bd09-4102-8d94-8ddffbfb5020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:56.199585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df53b0-bd09-4102-8d94-8ddffbfb5020 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:56.209778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 769/1545] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:05.599092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=37 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:06.204758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=42 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:27.043320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab4ed-23ca-42c0-b041-5ac7c1fda6ef None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.051474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab4ed-23ca-42c0-b041-5ac7c1fda6ef None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.051474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eab4ed-23ca-42c0-b041-5ac7c1fda6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.078958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-87eab4ed-23ca-42c0-b041-5ac7c1fda6ef None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.080743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-87eab4ed-23ca-42c0-b041-5ac7c1fda6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.091453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 777/1546] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:27.101541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6783c5-da21-44e7-8588-7f4ed4087ceb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.102160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6783c5-da21-44e7-8588-7f4ed4087ceb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.102160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6783c5-da21-44e7-8588-7f4ed4087ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.128246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-eb6783c5-da21-44e7-8588-7f4ed4087ceb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.129611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-eb6783c5-da21-44e7-8588-7f4ed4087ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.147383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 770/1547] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.152446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eab4ed-23ca-42c0-b041-5ac7c1fda6ef None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.167027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eab4ed-23ca-42c0-b041-5ac7c1fda6ef None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.173876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.174753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.175255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.175751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.176903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:27.177452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.180273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.195990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 778/1548] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:27.204846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6783c5-da21-44e7-8588-7f4ed4087ceb None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.216371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6783c5-da21-44e7-8588-7f4ed4087ceb None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.232849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dace4-57fe-4b14-abf2-5ccfe144ce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.233495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dace4-57fe-4b14-abf2-5ccfe144ce6a admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.233776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dace4-57fe-4b14-abf2-5ccfe144ce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.234046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dace4-57fe-4b14-abf2-5ccfe144ce6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.234721 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dace4-57fe-4b14-abf2-5ccfe144ce6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.236725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dace4-57fe-4b14-abf2-5ccfe144ce6a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.292261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 771/1549] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/projects => generated 365 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:27.297890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.314584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de2a41-71bc-4073-a3a1-a7f271b34d67 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.399068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d013d9-50c0-406b-b0dd-5c7d63f06f23 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.399546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d013d9-50c0-406b-b0dd-5c7d63f06f23 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.403484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d013d9-50c0-406b-b0dd-5c7d63f06f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.406908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e1a03-23ad-4f0b-952e-f0342fc215b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.407916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e1a03-23ad-4f0b-952e-f0342fc215b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.408443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e1a03-23ad-4f0b-952e-f0342fc215b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.408963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e1a03-23ad-4f0b-952e-f0342fc215b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.409895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e1a03-23ad-4f0b-952e-f0342fc215b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.412736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e1a03-23ad-4f0b-952e-f0342fc215b5 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.438101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e5d013d9-50c0-406b-b0dd-5c7d63f06f23 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.439359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e5d013d9-50c0-406b-b0dd-5c7d63f06f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.452969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 772/1550] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.455233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 779/1551] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:27.460623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d013d9-50c0-406b-b0dd-5c7d63f06f23 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.467992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d6056-2d7a-4538-85c8-100ebadce8d3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.468766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d6056-2d7a-4538-85c8-100ebadce8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.469319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d6056-2d7a-4538-85c8-100ebadce8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.476785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d013d9-50c0-406b-b0dd-5c7d63f06f23 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.484585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53208728-d4d4-479a-81b2-1eb947a9562b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.485185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53208728-d4d4-479a-81b2-1eb947a9562b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.485463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53208728-d4d4-479a-81b2-1eb947a9562b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.485734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53208728-d4d4-479a-81b2-1eb947a9562b admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.486376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53208728-d4d4-479a-81b2-1eb947a9562b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.488187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53208728-d4d4-479a-81b2-1eb947a9562b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.513275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 773/1552] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:27.513275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ca65-7b29-4422-ab5a-4db7c3f70c03 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.513275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ca65-7b29-4422-ab5a-4db7c3f70c03 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.513275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74ca65-7b29-4422-ab5a-4db7c3f70c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.515994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b75d6056-2d7a-4538-85c8-100ebadce8d3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.517474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b75d6056-2d7a-4538-85c8-100ebadce8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.531521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-db74ca65-7b29-4422-ab5a-4db7c3f70c03 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.532819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-db74ca65-7b29-4422-ab5a-4db7c3f70c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.537422 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 780/1553] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:27.539087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ead201-d7f8-4747-8eb7-9c0e910b4e0c None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.539593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ead201-d7f8-4747-8eb7-9c0e910b4e0c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.540417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ead201-d7f8-4747-8eb7-9c0e910b4e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.548553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 774/1554] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.548553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74ca65-7b29-4422-ab5a-4db7c3f70c03 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.566309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74ca65-7b29-4422-ab5a-4db7c3f70c03 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.580599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7498e44-9e07-46d6-be2f-577ab79f346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.580599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7498e44-9e07-46d6-be2f-577ab79f346d admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.580599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7498e44-9e07-46d6-be2f-577ab79f346d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.580599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7498e44-9e07-46d6-be2f-577ab79f346d admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.581080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7498e44-9e07-46d6-be2f-577ab79f346d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.581080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7498e44-9e07-46d6-be2f-577ab79f346d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.587551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-83ead201-d7f8-4747-8eb7-9c0e910b4e0c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.589087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-83ead201-d7f8-4747-8eb7-9c0e910b4e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.598317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 775/1555] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:27.599801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7498e44-9e07-46d6-be2f-577ab79f346d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.610545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 781/1556] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.612120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c58e4-f1eb-41cc-aa2a-6af41e320129 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.612634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c58e4-f1eb-41cc-aa2a-6af41e320129 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.613116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c58e4-f1eb-41cc-aa2a-6af41e320129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.624475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7498e44-9e07-46d6-be2f-577ab79f346d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.645703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.646451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.647019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.647405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8f1c58e4-f1eb-41cc-aa2a-6af41e320129 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.647614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.648606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:27.649125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.649965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8f1c58e4-f1eb-41cc-aa2a-6af41e320129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.651040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:17:27.651040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Aug 31 19:17:27.651040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: Aug 31 19:17:27.652592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.661889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 782/1557] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.663649 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98d3da-fc22-4683-a03f-c32136db927a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.664148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98d3da-fc22-4683-a03f-c32136db927a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.664638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98d3da-fc22-4683-a03f-c32136db927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.689664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 776/1558] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:27.690908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.694571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8c98d3da-fc22-4683-a03f-c32136db927a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.701363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8c98d3da-fc22-4683-a03f-c32136db927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.705927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fda7f-b711-40fd-b3e9-6f6397303438 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.711142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64032d1e-62ef-4048-828a-f2d8deb71091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.711842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64032d1e-62ef-4048-828a-f2d8deb71091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.712199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64032d1e-62ef-4048-828a-f2d8deb71091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.712388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64032d1e-62ef-4048-828a-f2d8deb71091 admin admin] PATH_INFO: `/v3/projects/e5d6fc02c5ab40cba5d60a67717786e5/users/02363f22cad04443a08d7fc03f8be3db/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.718234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 783/1559] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.720436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abcbe0-93eb-4a10-b146-068bd00644e7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.720981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abcbe0-93eb-4a10-b146-068bd00644e7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.721482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abcbe0-93eb-4a10-b146-068bd00644e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.742419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64032d1e-62ef-4048-828a-f2d8deb71091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d6fc02c5ab40cba5d60a67717786e5, user_id=02363f22cad04443a08d7fc03f8be3db, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.744908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-01abcbe0-93eb-4a10-b146-068bd00644e7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.745629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64032d1e-62ef-4048-828a-f2d8deb71091 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.746132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-01abcbe0-93eb-4a10-b146-068bd00644e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.755008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 784/1560] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.757087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abcbe0-93eb-4a10-b146-068bd00644e7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.767515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abcbe0-93eb-4a10-b146-068bd00644e7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.773429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 777/1561] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:27 2022] PUT /identity/v3/projects/e5d6fc02c5ab40cba5d60a67717786e5/users/02363f22cad04443a08d7fc03f8be3db/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:27.774753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9c2f-da70-4553-ad5f-eae18f1edd71 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.775043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9c2f-da70-4553-ad5f-eae18f1edd71 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.775323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9c2f-da70-4553-ad5f-eae18f1edd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.798365 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cc6f9c2f-da70-4553-ad5f-eae18f1edd71 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.799766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cc6f9c2f-da70-4553-ad5f-eae18f1edd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.832103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.832747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.833540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.833540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.833925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:27.834325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.836177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.847760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 785/1562] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:27.848385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.859925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9df5-3206-423f-a4f4-bc1183d86f82 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.863168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 778/1563] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.864582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e28c-d021-4efb-ad5e-3f50351eb980 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.864751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0521f-9ca8-4513-bf6f-22fdd303c89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.864949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e28c-d021-4efb-ad5e-3f50351eb980 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.865115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e28c-d021-4efb-ad5e-3f50351eb980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.865259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0521f-9ca8-4513-bf6f-22fdd303c89f admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.866071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0521f-9ca8-4513-bf6f-22fdd303c89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.866341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0521f-9ca8-4513-bf6f-22fdd303c89f admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.867170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0521f-9ca8-4513-bf6f-22fdd303c89f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.868211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING py.warnings [None req-6bc0521f-9ca8-4513-bf6f-22fdd303c89f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:17:27.868211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: warnings.warn(msg) Aug 31 19:17:27.868211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: Aug 31 19:17:27.869432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0521f-9ca8-4513-bf6f-22fdd303c89f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:27.889575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fcc3e28c-d021-4efb-ad5e-3f50351eb980 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.890760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fcc3e28c-d021-4efb-ad5e-3f50351eb980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.900133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 786/1564] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:27.900390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 779/1565] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:27.901835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e3b5-fea8-4290-a313-c10122c49a01 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.902226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e3b5-fea8-4290-a313-c10122c49a01 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.902449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3e28c-d021-4efb-ad5e-3f50351eb980 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:27.902583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e3b5-fea8-4290-a313-c10122c49a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.915125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3e28c-d021-4efb-ad5e-3f50351eb980 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:27.925227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-3121e3b5-fea8-4290-a313-c10122c49a01 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:27.926455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3121e3b5-fea8-4290-a313-c10122c49a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:27.932957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:27.961724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:27.981686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f07239-9ab0-43be-bcea-3bcdf6829cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:27.982307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07239-9ab0-43be-bcea-3bcdf6829cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:27.982591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07239-9ab0-43be-bcea-3bcdf6829cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:27.982871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07239-9ab0-43be-bcea-3bcdf6829cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:27.983523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f07239-9ab0-43be-bcea-3bcdf6829cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:27.983831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f07239-9ab0-43be-bcea-3bcdf6829cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:27.985585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f07239-9ab0-43be-bcea-3bcdf6829cd4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.005680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 780/1566] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:28.005680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535d9f2-6bea-4e4a-922e-11fd2595f8da None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.006066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 787/1567] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:28.006174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535d9f2-6bea-4e4a-922e-11fd2595f8da None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.006174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535d9f2-6bea-4e4a-922e-11fd2595f8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.006409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c58ca6-944b-48e7-92ba-a7ae8fb11a5f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.006409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c58ca6-944b-48e7-92ba-a7ae8fb11a5f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.006409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c58ca6-944b-48e7-92ba-a7ae8fb11a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.023961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1535d9f2-6bea-4e4a-922e-11fd2595f8da None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:28.025468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1535d9f2-6bea-4e4a-922e-11fd2595f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:28.033064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c5c58ca6-944b-48e7-92ba-a7ae8fb11a5f None None] MFA Rules not processed for user `02363f22cad04443a08d7fc03f8be3db`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:28.033332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 781/1568] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:28.034101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535d9f2-6bea-4e4a-922e-11fd2595f8da None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.034316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c5c58ca6-944b-48e7-92ba-a7ae8fb11a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:28.045759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535d9f2-6bea-4e4a-922e-11fd2595f8da None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.051686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83632f2a-63c3-4244-8d7a-ad8353f8d4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.052276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83632f2a-63c3-4244-8d7a-ad8353f8d4e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.052552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83632f2a-63c3-4244-8d7a-ad8353f8d4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.053573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83632f2a-63c3-4244-8d7a-ad8353f8d4e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.054428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83632f2a-63c3-4244-8d7a-ad8353f8d4e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.056681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83632f2a-63c3-4244-8d7a-ad8353f8d4e7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.085094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 782/1569] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.086546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae881962-26f7-411f-9a7f-80f826203cd3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.086780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae881962-26f7-411f-9a7f-80f826203cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.087123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae881962-26f7-411f-9a7f-80f826203cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.104474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 788/1570] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:28.105819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8fb58-3658-4f97-8e9f-f2067345bc10 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.106104 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8fb58-3658-4f97-8e9f-f2067345bc10 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.106374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8fb58-3658-4f97-8e9f-f2067345bc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.107092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ae881962-26f7-411f-9a7f-80f826203cd3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:28.108392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ae881962-26f7-411f-9a7f-80f826203cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:28.129832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-edc8fb58-3658-4f97-8e9f-f2067345bc10 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:28.131131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-edc8fb58-3658-4f97-8e9f-f2067345bc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:28.174085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 783/1571] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:28.174842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae881962-26f7-411f-9a7f-80f826203cd3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.184739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae881962-26f7-411f-9a7f-80f826203cd3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.190063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.190775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.191061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.191331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.191965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.193179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:17:28.193179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Aug 31 19:17:28.193179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: Aug 31 19:17:28.194425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.194915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 789/1572] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:28.195780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8fb58-3658-4f97-8e9f-f2067345bc10 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.205499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8fb58-3658-4f97-8e9f-f2067345bc10 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.210590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83085846-f8f9-40d0-8c56-cda2784cfaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.211165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085846-f8f9-40d0-8c56-cda2784cfaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.211514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085846-f8f9-40d0-8c56-cda2784cfaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.211790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085846-f8f9-40d0-8c56-cda2784cfaf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.212468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83085846-f8f9-40d0-8c56-cda2784cfaf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:28.212803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83085846-f8f9-40d0-8c56-cda2784cfaf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.214430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83085846-f8f9-40d0-8c56-cda2784cfaf4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.217110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 784/1573] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.217773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.227347 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 790/1574] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:28.228651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a58b3b-3592-4b6d-b179-a4c0cd141bbf None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.228937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a58b3b-3592-4b6d-b179-a4c0cd141bbf None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.229207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a58b3b-3592-4b6d-b179-a4c0cd141bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.230077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3dad8-eaec-4492-ac9c-9b54aa095bdb admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.239320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa07b8-d105-4d87-90a9-560ba3f6abf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.239948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa07b8-d105-4d87-90a9-560ba3f6abf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.240218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa07b8-d105-4d87-90a9-560ba3f6abf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.241112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa07b8-d105-4d87-90a9-560ba3f6abf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.241766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa07b8-d105-4d87-90a9-560ba3f6abf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.244557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa07b8-d105-4d87-90a9-560ba3f6abf8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.250457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c3a58b3b-3592-4b6d-b179-a4c0cd141bbf None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:28.251660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c3a58b3b-3592-4b6d-b179-a4c0cd141bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:28.259642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 791/1575] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:28.260432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a58b3b-3592-4b6d-b179-a4c0cd141bbf None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.270730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a58b3b-3592-4b6d-b179-a4c0cd141bbf None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.277148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fd899-4df5-4a15-9640-fb7d473ec832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.277650 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 785/1576] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.277799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fd899-4df5-4a15-9640-fb7d473ec832 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.278015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fd899-4df5-4a15-9640-fb7d473ec832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.278280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fd899-4df5-4a15-9640-fb7d473ec832 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.278494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa07b8-d105-4d87-90a9-560ba3f6abf8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.278914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fd899-4df5-4a15-9640-fb7d473ec832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.280989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fd899-4df5-4a15-9640-fb7d473ec832 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.305729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 792/1577] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.306344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fd899-4df5-4a15-9640-fb7d473ec832 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.311331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:28.316129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fd899-4df5-4a15-9640-fb7d473ec832 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.341029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa07b8-d105-4d87-90a9-560ba3f6abf8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.343812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65d93b-81cd-4f8a-b2c9-3526e4b037d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.344415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65d93b-81cd-4f8a-b2c9-3526e4b037d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.344723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65d93b-81cd-4f8a-b2c9-3526e4b037d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.345318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65d93b-81cd-4f8a-b2c9-3526e4b037d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.372253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:28.378546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.379161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.379441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.379714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.380358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:28.380787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.382752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.394190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 793/1578] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:28.394998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.405324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1eb4-eb73-4214-910e-f93b78ea90f1 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.410249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.410854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.411129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.411399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.412071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:28.413118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.414777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.421296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65d93b-81cd-4f8a-b2c9-3526e4b037d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.422759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65d93b-81cd-4f8a-b2c9-3526e4b037d6 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.427985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 794/1579] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:28.429494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.439259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7844fd-aee4-49f5-ae2a-3ce8a8f4ece5 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.440142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 786/1580] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:28.440886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65d93b-81cd-4f8a-b2c9-3526e4b037d6 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.445136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3168d-1159-4dcf-96ed-d0514e4a9a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.446397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3168d-1159-4dcf-96ed-d0514e4a9a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.446889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3168d-1159-4dcf-96ed-d0514e4a9a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.447166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3168d-1159-4dcf-96ed-d0514e4a9a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.447869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3168d-1159-4dcf-96ed-d0514e4a9a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.449508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a3168d-1159-4dcf-96ed-d0514e4a9a64 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.452849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65d93b-81cd-4f8a-b2c9-3526e4b037d6 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.458728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfb12-e9b2-4221-a026-3f4f248d2c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.460268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfb12-e9b2-4221-a026-3f4f248d2c0e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.461148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfb12-e9b2-4221-a026-3f4f248d2c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.465078 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 795/1581] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:28.465230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfb12-e9b2-4221-a026-3f4f248d2c0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.465318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3168d-1159-4dcf-96ed-d0514e4a9a64 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.465397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cfb12-e9b2-4221-a026-3f4f248d2c0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.465397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cfb12-e9b2-4221-a026-3f4f248d2c0e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.474213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3168d-1159-4dcf-96ed-d0514e4a9a64 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.479917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.480515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.480815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.481083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.481754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:28.482021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.483639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.496508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 787/1582] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.497688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfb12-e9b2-4221-a026-3f4f248d2c0e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.498225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 796/1583] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:28.499311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.516271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec933a-bdb0-4da5-8389-47813628fb90 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.578363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfb12-e9b2-4221-a026-3f4f248d2c0e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.583142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc597d7-8b19-400c-97ce-c1601c020d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.583645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4f1a7-9f4d-4d15-bf81-fa57184e6595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.583776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc597d7-8b19-400c-97ce-c1601c020d30 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.584021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc597d7-8b19-400c-97ce-c1601c020d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.585117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc597d7-8b19-400c-97ce-c1601c020d30 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.585242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4f1a7-9f4d-4d15-bf81-fa57184e6595 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.585716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4f1a7-9f4d-4d15-bf81-fa57184e6595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.585890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc597d7-8b19-400c-97ce-c1601c020d30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.586156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4f1a7-9f4d-4d15-bf81-fa57184e6595 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.586892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4f1a7-9f4d-4d15-bf81-fa57184e6595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.588446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc597d7-8b19-400c-97ce-c1601c020d30 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.588646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4f1a7-9f4d-4d15-bf81-fa57184e6595 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.602862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 797/1584] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:28.607340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4f1a7-9f4d-4d15-bf81-fa57184e6595 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.616769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4f1a7-9f4d-4d15-bf81-fa57184e6595 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.620684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 788/1585] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.620684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc597d7-8b19-400c-97ce-c1601c020d30 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.624711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1035-4ba1-4cf0-b2da-91de2d1c7a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.625710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1035-4ba1-4cf0-b2da-91de2d1c7a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.626122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1035-4ba1-4cf0-b2da-91de2d1c7a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.626533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1035-4ba1-4cf0-b2da-91de2d1c7a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.627192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c1035-4ba1-4cf0-b2da-91de2d1c7a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.628745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc597d7-8b19-400c-97ce-c1601c020d30 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.629185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c1035-4ba1-4cf0-b2da-91de2d1c7a06 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.636217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b500511-1d4c-4516-95c2-6da4fe1a45a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.637044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b500511-1d4c-4516-95c2-6da4fe1a45a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.637275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b500511-1d4c-4516-95c2-6da4fe1a45a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.637543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b500511-1d4c-4516-95c2-6da4fe1a45a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.638176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b500511-1d4c-4516-95c2-6da4fe1a45a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.640660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b500511-1d4c-4516-95c2-6da4fe1a45a9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.664037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 798/1586] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.664788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1035-4ba1-4cf0-b2da-91de2d1c7a06 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.679259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 789/1587] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.680102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b500511-1d4c-4516-95c2-6da4fe1a45a9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.707995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:28.742149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1035-4ba1-4cf0-b2da-91de2d1c7a06 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.761005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b500511-1d4c-4516-95c2-6da4fe1a45a9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.765956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b610a7-37e5-4123-b67d-0c8b0c59cf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.766627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b610a7-37e5-4123-b67d-0c8b0c59cf60 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.766914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b610a7-37e5-4123-b67d-0c8b0c59cf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.767195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b610a7-37e5-4123-b67d-0c8b0c59cf60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.767912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b610a7-37e5-4123-b67d-0c8b0c59cf60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.769975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b610a7-37e5-4123-b67d-0c8b0c59cf60 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.786835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:28.803143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 790/1588] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.804150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b610a7-37e5-4123-b67d-0c8b0c59cf60 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.813254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b717b-842d-466d-b39b-c13884a97360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.813996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b717b-842d-466d-b39b-c13884a97360 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.814361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b717b-842d-466d-b39b-c13884a97360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.814703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b717b-842d-466d-b39b-c13884a97360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.815475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b717b-842d-466d-b39b-c13884a97360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.817220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b717b-842d-466d-b39b-c13884a97360 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.832088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 799/1589] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:28.832088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b717b-842d-466d-b39b-c13884a97360 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.843609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b717b-842d-466d-b39b-c13884a97360 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.876490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b610a7-37e5-4123-b67d-0c8b0c59cf60 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.886107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd87fe6-b3dc-4bab-8702-e72ff88f5f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.886780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd87fe6-b3dc-4bab-8702-e72ff88f5f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.887066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd87fe6-b3dc-4bab-8702-e72ff88f5f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.887361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd87fe6-b3dc-4bab-8702-e72ff88f5f85 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.888032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd87fe6-b3dc-4bab-8702-e72ff88f5f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.890104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd87fe6-b3dc-4bab-8702-e72ff88f5f85 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.908988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3f9c4-a0f5-4011-a920-36d2913bb4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.909691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3f9c4-a0f5-4011-a920-36d2913bb4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.910045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3f9c4-a0f5-4011-a920-36d2913bb4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.910398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3f9c4-a0f5-4011-a920-36d2913bb4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.911133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3f9c4-a0f5-4011-a920-36d2913bb4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.913098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3f9c4-a0f5-4011-a920-36d2913bb4ce admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.915463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 791/1590] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:28.916135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd87fe6-b3dc-4bab-8702-e72ff88f5f85 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.928058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd87fe6-b3dc-4bab-8702-e72ff88f5f85 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.932407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 800/1591] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:28.932407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3f9c4-a0f5-4011-a920-36d2913bb4ce admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.941182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e5721-c701-439d-93b9-852995eaea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.941360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5721-c701-439d-93b9-852995eaea7c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.941623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5721-c701-439d-93b9-852995eaea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.941966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5721-c701-439d-93b9-852995eaea7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.945303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e5721-c701-439d-93b9-852995eaea7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.948763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e5721-c701-439d-93b9-852995eaea7c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.950983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3f9c4-a0f5-4011-a920-36d2913bb4ce admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.959958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cdc55-13d6-4d97-a61e-ef1d42353e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.960758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdc55-13d6-4d97-a61e-ef1d42353e71 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.961044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdc55-13d6-4d97-a61e-ef1d42353e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.961362 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdc55-13d6-4d97-a61e-ef1d42353e71 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.962307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cdc55-13d6-4d97-a61e-ef1d42353e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.962663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 792/1592] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:28.963508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e5721-c701-439d-93b9-852995eaea7c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:28.966460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cdc55-13d6-4d97-a61e-ef1d42353e71 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:28.973890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e5721-c701-439d-93b9-852995eaea7c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:28.979721 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707915f-c5ea-4bd2-9efa-827b7e23166c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:28.980430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707915f-c5ea-4bd2-9efa-827b7e23166c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:28.980765 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707915f-c5ea-4bd2-9efa-827b7e23166c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:28.981046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707915f-c5ea-4bd2-9efa-827b7e23166c admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:28.981778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707915f-c5ea-4bd2-9efa-827b7e23166c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:28.984144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707915f-c5ea-4bd2-9efa-827b7e23166c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.001169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 801/1593] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/users => generated 296 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:29.003817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cdc55-13d6-4d97-a61e-ef1d42353e71 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.015746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cdc55-13d6-4d97-a61e-ef1d42353e71 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.016269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 793/1594] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:28 2022] POST /identity/v3/users => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:29.017276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707915f-c5ea-4bd2-9efa-827b7e23166c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.023504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164b985-9ff3-4b76-adae-7a1efaec846e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.024396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164b985-9ff3-4b76-adae-7a1efaec846e admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.024907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164b985-9ff3-4b76-adae-7a1efaec846e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.025473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164b985-9ff3-4b76-adae-7a1efaec846e admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.026296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164b985-9ff3-4b76-adae-7a1efaec846e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.028789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164b985-9ff3-4b76-adae-7a1efaec846e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.031356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707915f-c5ea-4bd2-9efa-827b7e23166c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.038823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb523cb-7acb-4f54-ab98-c5d51db47202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.040124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb523cb-7acb-4f54-ab98-c5d51db47202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.040980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb523cb-7acb-4f54-ab98-c5d51db47202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.041324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb523cb-7acb-4f54-ab98-c5d51db47202 admin admin] PATH_INFO: `/v3/projects/9ccec772d24645ce8f5e58de8ee5e47c/users/97c9201758884fdbb080f85337a5ce87/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.063222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 802/1595] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:29 2022] POST /identity/v3/users => generated 287 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:29.064260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164b985-9ff3-4b76-adae-7a1efaec846e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.070545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb523cb-7acb-4f54-ab98-c5d51db47202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ccec772d24645ce8f5e58de8ee5e47c, user_id=97c9201758884fdbb080f85337a5ce87, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.073670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb523cb-7acb-4f54-ab98-c5d51db47202 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.085054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164b985-9ff3-4b76-adae-7a1efaec846e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.088845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 794/1596] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:29 2022] PUT /identity/v3/projects/9ccec772d24645ce8f5e58de8ee5e47c/users/97c9201758884fdbb080f85337a5ce87/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:29.089447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb523cb-7acb-4f54-ab98-c5d51db47202 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.130360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:29.161484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb523cb-7acb-4f54-ab98-c5d51db47202 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.162023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d867954-a575-4217-9ec7-f88916486a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.163117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867954-a575-4217-9ec7-f88916486a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.163117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867954-a575-4217-9ec7-f88916486a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.163464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d867954-a575-4217-9ec7-f88916486a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.164698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d867954-a575-4217-9ec7-f88916486a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.166723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d867954-a575-4217-9ec7-f88916486a5d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.167806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519a759-26d6-4693-b130-c6c715b1cfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.168651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519a759-26d6-4693-b130-c6c715b1cfc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.168934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519a759-26d6-4693-b130-c6c715b1cfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.169256 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519a759-26d6-4693-b130-c6c715b1cfc7 admin admin] PATH_INFO: `/v3/projects/1cb8f75600714e23bda1c8886bdb22f1/users/6e09978ed8b3490db5f92823768f1d15/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.179268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 803/1597] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.180272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d867954-a575-4217-9ec7-f88916486a5d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.186813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519a759-26d6-4693-b130-c6c715b1cfc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb8f75600714e23bda1c8886bdb22f1, user_id=6e09978ed8b3490db5f92823768f1d15, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.189885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519a759-26d6-4693-b130-c6c715b1cfc7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.190275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d867954-a575-4217-9ec7-f88916486a5d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.196268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2848-db16-47bb-b1a5-17a070e46313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.196987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2848-db16-47bb-b1a5-17a070e46313 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.197323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2848-db16-47bb-b1a5-17a070e46313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.197596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2848-db16-47bb-b1a5-17a070e46313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.198302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d2848-db16-47bb-b1a5-17a070e46313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.200077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d2848-db16-47bb-b1a5-17a070e46313 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.206174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 795/1598] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:29 2022] PUT /identity/v3/projects/1cb8f75600714e23bda1c8886bdb22f1/users/6e09978ed8b3490db5f92823768f1d15/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:29.207005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519a759-26d6-4693-b130-c6c715b1cfc7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.214107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 804/1599] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.214915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2848-db16-47bb-b1a5-17a070e46313 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.226781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2848-db16-47bb-b1a5-17a070e46313 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.231058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:29.278575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519a759-26d6-4693-b130-c6c715b1cfc7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.283514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28038-2327-4220-9e3c-b4a778152c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.284112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28038-2327-4220-9e3c-b4a778152c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.284391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28038-2327-4220-9e3c-b4a778152c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.284700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e28038-2327-4220-9e3c-b4a778152c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.285322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e28038-2327-4220-9e3c-b4a778152c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.286922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e28038-2327-4220-9e3c-b4a778152c04 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.287223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab1e37-9a92-47ed-a13f-f5b71bfe8196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.288132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab1e37-9a92-47ed-a13f-f5b71bfe8196 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.288420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab1e37-9a92-47ed-a13f-f5b71bfe8196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.288759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab1e37-9a92-47ed-a13f-f5b71bfe8196 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.289412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab1e37-9a92-47ed-a13f-f5b71bfe8196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.291295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab1e37-9a92-47ed-a13f-f5b71bfe8196 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.297345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 796/1600] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.298309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28038-2327-4220-9e3c-b4a778152c04 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.305089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 805/1601] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.306164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fecb-ea8d-4e5b-9324-84107e8203d2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.306452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fecb-ea8d-4e5b-9324-84107e8203d2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.306726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896fecb-ea8d-4e5b-9324-84107e8203d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.311682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e28038-2327-4220-9e3c-b4a778152c04 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.317432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a34cfd-9cee-455e-bca0-1b4038084697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.318034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a34cfd-9cee-455e-bca0-1b4038084697 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.318307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a34cfd-9cee-455e-bca0-1b4038084697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.319151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a34cfd-9cee-455e-bca0-1b4038084697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.320154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a34cfd-9cee-455e-bca0-1b4038084697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.323807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a34cfd-9cee-455e-bca0-1b4038084697 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.328621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b896fecb-ea8d-4e5b-9324-84107e8203d2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:29.329876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b896fecb-ea8d-4e5b-9324-84107e8203d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:29.339564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 797/1602] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.339564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a34cfd-9cee-455e-bca0-1b4038084697 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.339836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 806/1603] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:29.339836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09491a1-fe63-48f7-b5ad-7b3448f93bc2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.340118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09491a1-fe63-48f7-b5ad-7b3448f93bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.340396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09491a1-fe63-48f7-b5ad-7b3448f93bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.351641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a34cfd-9cee-455e-bca0-1b4038084697 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.356220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acb940-a0ae-44da-ba9f-d68fe85fa2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.356850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acb940-a0ae-44da-ba9f-d68fe85fa2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.357126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acb940-a0ae-44da-ba9f-d68fe85fa2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.357397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acb940-a0ae-44da-ba9f-d68fe85fa2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.358035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acb940-a0ae-44da-ba9f-d68fe85fa2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.359628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acb940-a0ae-44da-ba9f-d68fe85fa2f5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.360463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b09491a1-fe63-48f7-b5ad-7b3448f93bc2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:29.362019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b09491a1-fe63-48f7-b5ad-7b3448f93bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:29.370998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 798/1604] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.373282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acb940-a0ae-44da-ba9f-d68fe85fa2f5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.376724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 807/1605] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:29.377599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09491a1-fe63-48f7-b5ad-7b3448f93bc2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.387216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acb940-a0ae-44da-ba9f-d68fe85fa2f5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.388485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09491a1-fe63-48f7-b5ad-7b3448f93bc2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.393491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce059d06-92ff-4792-ab58-2fa3eaf07046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.394080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282a4aa-91ab-4577-ba42-00d663259a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.394297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce059d06-92ff-4792-ab58-2fa3eaf07046 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.394584 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce059d06-92ff-4792-ab58-2fa3eaf07046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.394906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce059d06-92ff-4792-ab58-2fa3eaf07046 admin admin] PATH_INFO: `/v3/projects/4adeb4ab309d44a6b3440df802f5bb25/users/25c3f0a2e1fd4067b1804ef42faa46d5/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.395099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282a4aa-91ab-4577-ba42-00d663259a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.395274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282a4aa-91ab-4577-ba42-00d663259a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.395547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282a4aa-91ab-4577-ba42-00d663259a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.396503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282a4aa-91ab-4577-ba42-00d663259a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.398382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282a4aa-91ab-4577-ba42-00d663259a99 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.410903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 808/1606] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.412038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282a4aa-91ab-4577-ba42-00d663259a99 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.416011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce059d06-92ff-4792-ab58-2fa3eaf07046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4adeb4ab309d44a6b3440df802f5bb25, user_id=25c3f0a2e1fd4067b1804ef42faa46d5, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.418894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce059d06-92ff-4792-ab58-2fa3eaf07046 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.430027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282a4aa-91ab-4577-ba42-00d663259a99 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.435007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 799/1607] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:29 2022] PUT /identity/v3/projects/4adeb4ab309d44a6b3440df802f5bb25/users/25c3f0a2e1fd4067b1804ef42faa46d5/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:29.436014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce059d06-92ff-4792-ab58-2fa3eaf07046 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.502764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce059d06-92ff-4792-ab58-2fa3eaf07046 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.509564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b2c39-8d77-4781-a071-1419b8738855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.510330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b2c39-8d77-4781-a071-1419b8738855 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.510723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b2c39-8d77-4781-a071-1419b8738855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.511085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b2c39-8d77-4781-a071-1419b8738855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.511864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b2c39-8d77-4781-a071-1419b8738855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.513747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b2c39-8d77-4781-a071-1419b8738855 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.514331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b439b-f479-4611-a2a7-08d11fb9aa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.514950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b439b-f479-4611-a2a7-08d11fb9aa08 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.515229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b439b-f479-4611-a2a7-08d11fb9aa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.515503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b439b-f479-4611-a2a7-08d11fb9aa08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.516143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b439b-f479-4611-a2a7-08d11fb9aa08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.517767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b439b-f479-4611-a2a7-08d11fb9aa08 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.528305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 800/1608] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.528305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b2c39-8d77-4781-a071-1419b8738855 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.528622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 809/1609] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:29.528989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b439b-f479-4611-a2a7-08d11fb9aa08 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.539143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b439b-f479-4611-a2a7-08d11fb9aa08 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.544083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.544753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.545036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.545310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] PATH_INFO: `/v3/projects/1ecdbc1a962e47a2b1885519551b23e6/users/50cc345e23a44acfb5dcf189a56124b1/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.561439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecdbc1a962e47a2b1885519551b23e6, user_id=50cc345e23a44acfb5dcf189a56124b1, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.562551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING py.warnings [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:17:29.562551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: warnings.warn(msg) Aug 31 19:17:29.562551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: Aug 31 19:17:29.564604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.579874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 810/1610] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:29 2022] PUT /identity/v3/projects/1ecdbc1a962e47a2b1885519551b23e6/users/50cc345e23a44acfb5dcf189a56124b1/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:29.580994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.595076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b2c39-8d77-4781-a071-1419b8738855 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.602467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:29.610260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:29.652356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca08aa-0de3-49a9-8608-a51876660437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.653080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca08aa-0de3-49a9-8608-a51876660437 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.653362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca08aa-0de3-49a9-8608-a51876660437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.653465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53bdda-ecef-4a40-9ec0-2f0dcc8fe4ba admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.653631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca08aa-0de3-49a9-8608-a51876660437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.657103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e3a44-4900-4b95-b87e-d127037cfc1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.657715 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e3a44-4900-4b95-b87e-d127037cfc1a service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.657995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e3a44-4900-4b95-b87e-d127037cfc1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.658263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e3a44-4900-4b95-b87e-d127037cfc1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.721716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e3a44-4900-4b95-b87e-d127037cfc1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.722033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca08aa-0de3-49a9-8608-a51876660437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.723204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e3a44-4900-4b95-b87e-d127037cfc1a service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.723541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca08aa-0de3-49a9-8608-a51876660437 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.734413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 801/1611] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:29.734952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 811/1612] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:29.735240 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca08aa-0de3-49a9-8608-a51876660437 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.736277 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9b55a-12fb-4eae-8780-f40ab0c90edb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.736605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9b55a-12fb-4eae-8780-f40ab0c90edb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.737052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9b55a-12fb-4eae-8780-f40ab0c90edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.747426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca08aa-0de3-49a9-8608-a51876660437 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.753322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0000d-391d-47ce-bb34-35bc4cc76b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.754174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0000d-391d-47ce-bb34-35bc4cc76b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.754556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0000d-391d-47ce-bb34-35bc4cc76b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.755027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0000d-391d-47ce-bb34-35bc4cc76b6f admin admin] PATH_INFO: `/v3/projects/a1342c2dbb1c41dca6061505220d9de7/users/fd0f7f51482040d7a89df3f7502c2638/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.759643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-cfd9b55a-12fb-4eae-8780-f40ab0c90edb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:29.760562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cfd9b55a-12fb-4eae-8780-f40ab0c90edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:29.769092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 812/1613] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:29.770080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9b55a-12fb-4eae-8780-f40ab0c90edb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.776164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0000d-391d-47ce-bb34-35bc4cc76b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1342c2dbb1c41dca6061505220d9de7, user_id=fd0f7f51482040d7a89df3f7502c2638, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.778865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0000d-391d-47ce-bb34-35bc4cc76b6f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.783360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9b55a-12fb-4eae-8780-f40ab0c90edb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.790938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116b5e3-7b0b-4ede-83f6-d4c79157f721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:29.791618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116b5e3-7b0b-4ede-83f6-d4c79157f721 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.791901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116b5e3-7b0b-4ede-83f6-d4c79157f721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.792171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116b5e3-7b0b-4ede-83f6-d4c79157f721 admin admin] PATH_INFO: `/v3/projects/2fc2cc05aa8a4b1e8c729315f6c85256/users/782f913b34bf41c394642799049dc4ed/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.804678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 802/1614] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:29 2022] PUT /identity/v3/projects/a1342c2dbb1c41dca6061505220d9de7/users/fd0f7f51482040d7a89df3f7502c2638/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:29.806057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0000d-391d-47ce-bb34-35bc4cc76b6f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.812206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116b5e3-7b0b-4ede-83f6-d4c79157f721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fc2cc05aa8a4b1e8c729315f6c85256, user_id=782f913b34bf41c394642799049dc4ed, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:29.815516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116b5e3-7b0b-4ede-83f6-d4c79157f721 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:29.831933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 813/1615] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:29 2022] PUT /identity/v3/projects/2fc2cc05aa8a4b1e8c729315f6c85256/users/782f913b34bf41c394642799049dc4ed/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:29.833490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ac9e-0050-4e82-b847-c802953f1e7f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:29.833490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ac9e-0050-4e82-b847-c802953f1e7f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:29.833845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167ac9e-0050-4e82-b847-c802953f1e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:29.868285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b167ac9e-0050-4e82-b847-c802953f1e7f None None] MFA Rules not processed for user `50cc345e23a44acfb5dcf189a56124b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:29.869748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b167ac9e-0050-4e82-b847-c802953f1e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:29.889183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0000d-391d-47ce-bb34-35bc4cc76b6f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:29.964377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 814/1616] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:29.965686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167ac9e-0050-4e82-b847-c802953f1e7f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:29.971279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b167ac9e-0050-4e82-b847-c802953f1e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:29.979705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:30.000014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ac577-3f3f-4209-a876-f95e65629861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:30.001172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ac577-3f3f-4209-a876-f95e65629861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.002441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:30.002587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ac577-3f3f-4209-a876-f95e65629861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.003148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ac577-3f3f-4209-a876-f95e65629861 admin admin] PATH_INFO: `/v3/projects/6d1e607dfc7e4b089c7e744f4b1059e8/users/a288c513afd94adaa9f53f9f65dc9dd8/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.027453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ac577-3f3f-4209-a876-f95e65629861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d1e607dfc7e4b089c7e744f4b1059e8, user_id=a288c513afd94adaa9f53f9f65dc9dd8, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:30.030972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ac577-3f3f-4209-a876-f95e65629861 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:30.049693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 803/1617] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:29 2022] PUT /identity/v3/projects/6d1e607dfc7e4b089c7e744f4b1059e8/users/a288c513afd94adaa9f53f9f65dc9dd8/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:30.051720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de92a3-d366-4d14-929b-f44299b83302 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.052182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de92a3-d366-4d14-929b-f44299b83302 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.052683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de92a3-d366-4d14-929b-f44299b83302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.060652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167ac9e-0050-4e82-b847-c802953f1e7f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:30.076129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e0de92a3-d366-4d14-929b-f44299b83302 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:30.078345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e0de92a3-d366-4d14-929b-f44299b83302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:30.120259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968f2bb-cc16-4894-8ec5-84246ea45d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:30.120819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968f2bb-cc16-4894-8ec5-84246ea45d89 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.121094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968f2bb-cc16-4894-8ec5-84246ea45d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.121360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968f2bb-cc16-4894-8ec5-84246ea45d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.201391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 804/1618] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:30.202837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152124a-a69f-4aa2-8bb3-e6f3537eb95e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.203129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152124a-a69f-4aa2-8bb3-e6f3537eb95e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.203403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152124a-a69f-4aa2-8bb3-e6f3537eb95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.228703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968f2bb-cc16-4894-8ec5-84246ea45d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:30.230216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968f2bb-cc16-4894-8ec5-84246ea45d89 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:30.241431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8152124a-a69f-4aa2-8bb3-e6f3537eb95e None None] MFA Rules not processed for user `782f913b34bf41c394642799049dc4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:30.244332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 815/1619] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:30.246340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8152124a-a69f-4aa2-8bb3-e6f3537eb95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:30.246586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30ae1-d64a-464c-944d-e9751668374a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.246841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30ae1-d64a-464c-944d-e9751668374a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.247116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30ae1-d64a-464c-944d-e9751668374a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.248665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 816/1620] 10.222.0.95 () {54 vars in 962 bytes} [Wed Aug 31 19:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:30.249555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ee10-216a-4aa3-b611-430ccace5cb2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.249944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ee10-216a-4aa3-b611-430ccace5cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.250156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ee10-216a-4aa3-b611-430ccace5cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.287079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-84b9ee10-216a-4aa3-b611-430ccace5cb2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:30.289778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-84b9ee10-216a-4aa3-b611-430ccace5cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:30.320735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 817/1621] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:30.321899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9ee10-216a-4aa3-b611-430ccace5cb2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:30.352897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9ee10-216a-4aa3-b611-430ccace5cb2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:30.370844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14a6c2-87b4-4d06-8f54-ce425e40ab4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:30.371485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14a6c2-87b4-4d06-8f54-ce425e40ab4d service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.371791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14a6c2-87b4-4d06-8f54-ce425e40ab4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.372067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14a6c2-87b4-4d06-8f54-ce425e40ab4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.373181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 805/1622] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:30.377348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026c8b9-190c-4f23-8464-1a5f753f72c2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.377700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026c8b9-190c-4f23-8464-1a5f753f72c2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.378029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026c8b9-190c-4f23-8464-1a5f753f72c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.388139 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14a6c2-87b4-4d06-8f54-ce425e40ab4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:30.389647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14a6c2-87b4-4d06-8f54-ce425e40ab4d service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:30.406357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 818/1623] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:30.407867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14a6c2-87b4-4d06-8f54-ce425e40ab4d service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:30.419915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14a6c2-87b4-4d06-8f54-ce425e40ab4d service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:30.424135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0bee8-fa9f-4ae6-8d57-6bb873c7004a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:30.425122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0bee8-fa9f-4ae6-8d57-6bb873c7004a service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.425410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0bee8-fa9f-4ae6-8d57-6bb873c7004a service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.425684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0bee8-fa9f-4ae6-8d57-6bb873c7004a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.429247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7026c8b9-190c-4f23-8464-1a5f753f72c2 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:30.430593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7026c8b9-190c-4f23-8464-1a5f753f72c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:30.439018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0bee8-fa9f-4ae6-8d57-6bb873c7004a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:30.441005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0bee8-fa9f-4ae6-8d57-6bb873c7004a service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:30.455620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 819/1624] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:30.487518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:30.523703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3952a-979a-424e-88d8-975d6e5f0c6c None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.524095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3952a-979a-424e-88d8-975d6e5f0c6c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.524438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3952a-979a-424e-88d8-975d6e5f0c6c None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.526696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 820/1625] 10.222.0.95 () {54 vars in 962 bytes} [Wed Aug 31 19:17:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:17:30.535003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 806/1626] 10.222.0.95 () {58 vars in 1059 bytes} [Wed Aug 31 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:30.536176 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3952a-979a-424e-88d8-975d6e5f0c6c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:30.557344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80e99-dded-47c2-98f3-9dcadc7bf496 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.559421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80e99-dded-47c2-98f3-9dcadc7bf496 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.559883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e80e99-dded-47c2-98f3-9dcadc7bf496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.570894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3952a-979a-424e-88d8-975d6e5f0c6c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:30.582328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3dff44-4250-4330-8880-4ddc2d9fccbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:30.583049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dff44-4250-4330-8880-4ddc2d9fccbf service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.583492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dff44-4250-4330-8880-4ddc2d9fccbf service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.583812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3dff44-4250-4330-8880-4ddc2d9fccbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.601729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-02e80e99-dded-47c2-98f3-9dcadc7bf496 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:30.603262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-02e80e99-dded-47c2-98f3-9dcadc7bf496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:30.621984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 807/1627] 10.222.0.95 () {58 vars in 1059 bytes} [Wed Aug 31 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:30.623674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04feba-6e3b-47cb-88dc-438b5c2f6886 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.624027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04feba-6e3b-47cb-88dc-438b5c2f6886 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.624619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04feba-6e3b-47cb-88dc-438b5c2f6886 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.626286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 808/1628] 10.222.0.95 () {54 vars in 912 bytes} [Wed Aug 31 19:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:30.635173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d75610-82af-4be2-9401-5b0f2ec22783 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.635717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d75610-82af-4be2-9401-5b0f2ec22783 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.636087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d75610-82af-4be2-9401-5b0f2ec22783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.671361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b3d75610-82af-4be2-9401-5b0f2ec22783 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:30.672877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b3d75610-82af-4be2-9401-5b0f2ec22783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:30.698984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 809/1629] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:30.707558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d75610-82af-4be2-9401-5b0f2ec22783 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:30.710653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:30.729717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3dff44-4250-4330-8880-4ddc2d9fccbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:30.733482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3dff44-4250-4330-8880-4ddc2d9fccbf service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:30.773841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 821/1630] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:30.779658 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d75610-82af-4be2-9401-5b0f2ec22783 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:30.784815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f02fa3-8cb7-4155-96c6-1d8db03d6622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:30.785553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02fa3-8cb7-4155-96c6-1d8db03d6622 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.785929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02fa3-8cb7-4155-96c6-1d8db03d6622 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.786269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f02fa3-8cb7-4155-96c6-1d8db03d6622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.801212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f02fa3-8cb7-4155-96c6-1d8db03d6622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:30.802761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f02fa3-8cb7-4155-96c6-1d8db03d6622 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:30.832776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 810/1631] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:17:30.893231 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3dff44-4250-4330-8880-4ddc2d9fccbf service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:30.908962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3dff44-4250-4330-8880-4ddc2d9fccbf service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:30.928560 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48998c2-0ae8-40dd-a9a5-67b18cbfbb34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:30.929613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48998c2-0ae8-40dd-a9a5-67b18cbfbb34 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:30.930106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48998c2-0ae8-40dd-a9a5-67b18cbfbb34 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:30.930594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48998c2-0ae8-40dd-a9a5-67b18cbfbb34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:30.951224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48998c2-0ae8-40dd-a9a5-67b18cbfbb34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:30.953396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48998c2-0ae8-40dd-a9a5-67b18cbfbb34 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:30.974493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 822/1632] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:31.299153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5082cfa-2d2f-41b1-b97c-0a11e4db8285 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:31.300831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5082cfa-2d2f-41b1-b97c-0a11e4db8285 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:31.301648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5082cfa-2d2f-41b1-b97c-0a11e4db8285 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:31.306358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 811/1633] 10.222.0.95 () {54 vars in 912 bytes} [Wed Aug 31 19:17:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:31.312240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086f2a-e1ba-4223-abe8-fe58115fdde2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:31.313077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086f2a-e1ba-4223-abe8-fe58115fdde2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:31.313077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086f2a-e1ba-4223-abe8-fe58115fdde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:31.337315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-bb086f2a-e1ba-4223-abe8-fe58115fdde2 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:31.340663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-bb086f2a-e1ba-4223-abe8-fe58115fdde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:31.362755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 823/1634] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:17:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:31.408284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5082cfa-2d2f-41b1-b97c-0a11e4db8285 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:31.434371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5082cfa-2d2f-41b1-b97c-0a11e4db8285 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:31.448733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86085a-5cd8-4262-825b-cbbd1009d039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:31.449459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86085a-5cd8-4262-825b-cbbd1009d039 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:31.449870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86085a-5cd8-4262-825b-cbbd1009d039 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:31.450348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86085a-5cd8-4262-825b-cbbd1009d039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:31.480532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86085a-5cd8-4262-825b-cbbd1009d039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:31.484453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86085a-5cd8-4262-825b-cbbd1009d039 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:31.506141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 812/1635] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:31.517085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb086f2a-e1ba-4223-abe8-fe58115fdde2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:31.541994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb086f2a-e1ba-4223-abe8-fe58115fdde2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:31.550223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:31.551285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:31.551767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:31.552237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] PATH_INFO: `/v3/users/782f913b34bf41c394642799049dc4ed` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:31.554318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] RBAC: Authorizing `identity:delete_user(user_id=782f913b34bf41c394642799049dc4ed)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:31.556714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: WARNING py.warnings [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:17:31.556714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: warnings.warn(msg) Aug 31 19:17:31.556714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: Aug 31 19:17:31.568744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:31.784280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782f913b34bf41c394642799049dc4ed'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:31.825712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782f913b34bf41c394642799049dc4ed'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:31.829419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782f913b34bf41c394642799049dc4ed'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:31.855854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782f913b34bf41c394642799049dc4ed'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:31.856891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782f913b34bf41c394642799049dc4ed'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:31.890380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 824/1636] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:31 2022] DELETE /identity/v3/users/782f913b34bf41c394642799049dc4ed => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:31.892818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ebee-1fe8-46a6-94ae-a1830d0f314d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:31.893307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ebee-1fe8-46a6-94ae-a1830d0f314d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:31.893780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ebee-1fe8-46a6-94ae-a1830d0f314d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:31.943196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e8d7ebee-1fe8-46a6-94ae-a1830d0f314d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:31.946200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e8d7ebee-1fe8-46a6-94ae-a1830d0f314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:32.048076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 813/1637] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:32.063555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:32.065081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:32.221267 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288e46e-63a8-402a-a408-f77f95a0499f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:32.228950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497159e3-9ec6-462f-86a2-803554b947dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:32.229763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497159e3-9ec6-462f-86a2-803554b947dd service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:32.230345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497159e3-9ec6-462f-86a2-803554b947dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:32.230745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497159e3-9ec6-462f-86a2-803554b947dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:32.242109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497159e3-9ec6-462f-86a2-803554b947dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:32.244461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497159e3-9ec6-462f-86a2-803554b947dd service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:32.259098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 825/1638] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:35.914079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7ebee-1fe8-46a6-94ae-a1830d0f314d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:35.915353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e8d7ebee-1fe8-46a6-94ae-a1830d0f314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:35.934100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7ebee-1fe8-46a6-94ae-a1830d0f314d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:35.949302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:35.949959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:35.950400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:35.950692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] PATH_INFO: `/v3/projects/2fc2cc05aa8a4b1e8c729315f6c85256` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:35.952986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fc2cc05aa8a4b1e8c729315f6c85256)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:35.954232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:17:35.954232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Aug 31 19:17:35.954232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: Aug 31 19:17:35.955440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:36.106729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fc2cc05aa8a4b1e8c729315f6c85256', 'request_id': 'req-9b51934d-f5bb-48a0-9083-7d8943a5f361'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:36.129904 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:36.149242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fc2cc05aa8a4b1e8c729315f6c85256', 'request_id': 'req-9b51934d-f5bb-48a0-9083-7d8943a5f361'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:36.179486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fc2cc05aa8a4b1e8c729315f6c85256', 'request_id': 'req-9b51934d-f5bb-48a0-9083-7d8943a5f361'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:36.204366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] The token cache is being invalidate because project 2fc2cc05aa8a4b1e8c729315f6c85256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:17:36.204868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:36.205795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9b51934d-f5bb-48a0-9083-7d8943a5f361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:36.207868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 814/1639] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:17:35 2022] DELETE /identity/v3/projects/2fc2cc05aa8a4b1e8c729315f6c85256 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:36.215629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b371b-f0dd-47a7-82bd-b2c0c089e557 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.216063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b371b-f0dd-47a7-82bd-b2c0c089e557 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.216418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b371b-f0dd-47a7-82bd-b2c0c089e557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.242963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f30b371b-f0dd-47a7-82bd-b2c0c089e557 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:36.244632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f30b371b-f0dd-47a7-82bd-b2c0c089e557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:36.340698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:36.407607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 826/1640] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:36.408200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33909756-c462-4ba0-837e-f18be1cacfad None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.408528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33909756-c462-4ba0-837e-f18be1cacfad None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.408843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33909756-c462-4ba0-837e-f18be1cacfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.457489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-33909756-c462-4ba0-837e-f18be1cacfad None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:36.458855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-33909756-c462-4ba0-837e-f18be1cacfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:36.489956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 815/1641] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:36.495508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b371b-f0dd-47a7-82bd-b2c0c089e557 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:36.510769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b371b-f0dd-47a7-82bd-b2c0c089e557 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:36.520216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:36.521032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.522068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.522447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.523242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:36.523650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:36.526142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:36.551716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 827/1642] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:36.552228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33909756-c462-4ba0-837e-f18be1cacfad None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:36.562295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33909756-c462-4ba0-837e-f18be1cacfad None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:36.569244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb3d38-8022-43cc-a2c0-152465d6d1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:36.569552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb3d38-8022-43cc-a2c0-152465d6d1fa admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.569806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb3d38-8022-43cc-a2c0-152465d6d1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.570081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb3d38-8022-43cc-a2c0-152465d6d1fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.570841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb3d38-8022-43cc-a2c0-152465d6d1fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:36.572829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb3d38-8022-43cc-a2c0-152465d6d1fa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:36.612917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 816/1643] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:36 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:17:36.614557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:36.627014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05917754-af31-499c-8df8-0c8d7e4e98c6 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:36.714314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f365e0-cc70-411f-abbf-93fe0edc4274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:36.715557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f365e0-cc70-411f-abbf-93fe0edc4274 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.716135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f365e0-cc70-411f-abbf-93fe0edc4274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.716631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f365e0-cc70-411f-abbf-93fe0edc4274 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.717501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f365e0-cc70-411f-abbf-93fe0edc4274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:36.720224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f365e0-cc70-411f-abbf-93fe0edc4274 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:36.760339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb3d38-8022-43cc-a2c0-152465d6d1fa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:36.762012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 828/1644] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:36 2022] POST /identity/v3/users => generated 282 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:36.774460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb3d38-8022-43cc-a2c0-152465d6d1fa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:36.779410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2022e-4ec2-4e4d-8874-eaa38f7d8ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:36.780249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2022e-4ec2-4e4d-8874-eaa38f7d8ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.780361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2022e-4ec2-4e4d-8874-eaa38f7d8ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.780561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2022e-4ec2-4e4d-8874-eaa38f7d8ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.781610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2022e-4ec2-4e4d-8874-eaa38f7d8ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:36.783286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2022e-4ec2-4e4d-8874-eaa38f7d8ca7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:36.801133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 817/1645] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:36.804435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f365e0-cc70-411f-abbf-93fe0edc4274 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:36.826822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f365e0-cc70-411f-abbf-93fe0edc4274 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:36.836501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b4d97-4207-4196-88b8-be242deda43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:36.837423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b4d97-4207-4196-88b8-be242deda43b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.838456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b4d97-4207-4196-88b8-be242deda43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.838912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b4d97-4207-4196-88b8-be242deda43b admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.840206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b4d97-4207-4196-88b8-be242deda43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:36.844057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b4d97-4207-4196-88b8-be242deda43b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:36.866717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 829/1646] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:36.870376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2022e-4ec2-4e4d-8874-eaa38f7d8ca7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:36.901552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2022e-4ec2-4e4d-8874-eaa38f7d8ca7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:36.910086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960005d9-6140-400b-8290-9ba5bcb9162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:36.910959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960005d9-6140-400b-8290-9ba5bcb9162e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.911564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960005d9-6140-400b-8290-9ba5bcb9162e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.912614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960005d9-6140-400b-8290-9ba5bcb9162e admin admin] PATH_INFO: `/v3/projects/eafec54acec540ad9c9ed393e61b8707/users/af37589186124cff9cd005e3814c09dd/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:36.937078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960005d9-6140-400b-8290-9ba5bcb9162e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eafec54acec540ad9c9ed393e61b8707, user_id=af37589186124cff9cd005e3814c09dd, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:36.942404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960005d9-6140-400b-8290-9ba5bcb9162e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:36.970867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 818/1647] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:36 2022] PUT /identity/v3/projects/eafec54acec540ad9c9ed393e61b8707/users/af37589186124cff9cd005e3814c09dd/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:17:36.983177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e69753-1e0a-4b1e-92de-666424011cfb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:36.983757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e69753-1e0a-4b1e-92de-666424011cfb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:36.984324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e69753-1e0a-4b1e-92de-666424011cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:37.025002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e7e69753-1e0a-4b1e-92de-666424011cfb None None] MFA Rules not processed for user `af37589186124cff9cd005e3814c09dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:37.027612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e7e69753-1e0a-4b1e-92de-666424011cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:37.147822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 830/1648] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:37.149368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960005d9-6140-400b-8290-9ba5bcb9162e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:37.253132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960005d9-6140-400b-8290-9ba5bcb9162e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:37.258353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e4f8d-08e6-460a-af89-0a1e5332f008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:37.259364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e4f8d-08e6-460a-af89-0a1e5332f008 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:37.260188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e4f8d-08e6-460a-af89-0a1e5332f008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:37.260932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e4f8d-08e6-460a-af89-0a1e5332f008 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:37.261846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e4f8d-08e6-460a-af89-0a1e5332f008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:37.264166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e4f8d-08e6-460a-af89-0a1e5332f008 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:37.303248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 819/1649] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:37 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:37.307479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e69753-1e0a-4b1e-92de-666424011cfb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:37.333447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e69753-1e0a-4b1e-92de-666424011cfb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:37.449733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:37.533638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491867c-b8f7-4cdc-a344-79e927b9ca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:37.534659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491867c-b8f7-4cdc-a344-79e927b9ca0b admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:37.535467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491867c-b8f7-4cdc-a344-79e927b9ca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:37.535583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491867c-b8f7-4cdc-a344-79e927b9ca0b admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:37.537566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491867c-b8f7-4cdc-a344-79e927b9ca0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:37.546041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491867c-b8f7-4cdc-a344-79e927b9ca0b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:37.634234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 831/1650] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:37 2022] POST /identity/v3/users => generated 288 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:37.635251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e4f8d-08e6-460a-af89-0a1e5332f008 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:37.655476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e4f8d-08e6-460a-af89-0a1e5332f008 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:37.661820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e2992-6526-4284-a952-e969e6e6ca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:37.663061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2992-6526-4284-a952-e969e6e6ca8e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:37.663346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2992-6526-4284-a952-e969e6e6ca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:37.663622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2992-6526-4284-a952-e969e6e6ca8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:37.664292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e2992-6526-4284-a952-e969e6e6ca8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:37.666221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e2992-6526-4284-a952-e969e6e6ca8e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:37.701400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 820/1651] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:37.703913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491867c-b8f7-4cdc-a344-79e927b9ca0b admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:37.732427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491867c-b8f7-4cdc-a344-79e927b9ca0b admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:37.754021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ac0fe-7b42-4c53-b5cb-8d5d9643ffcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:37.755569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ac0fe-7b42-4c53-b5cb-8d5d9643ffcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:37.755569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ac0fe-7b42-4c53-b5cb-8d5d9643ffcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:37.755569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ac0fe-7b42-4c53-b5cb-8d5d9643ffcf admin admin] PATH_INFO: `/v3/projects/284ebd2544ae4c6d8e4ff13a4330c3f8/users/235894f779884573b8263c00954bf62d/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:37.790199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ac0fe-7b42-4c53-b5cb-8d5d9643ffcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=284ebd2544ae4c6d8e4ff13a4330c3f8, user_id=235894f779884573b8263c00954bf62d, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:37.793415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ac0fe-7b42-4c53-b5cb-8d5d9643ffcf admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:37.845641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 832/1652] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:37 2022] PUT /identity/v3/projects/284ebd2544ae4c6d8e4ff13a4330c3f8/users/235894f779884573b8263c00954bf62d/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:37.853146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e2992-6526-4284-a952-e969e6e6ca8e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:37.869903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e2992-6526-4284-a952-e969e6e6ca8e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:38.020139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:38.080073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379831e-23fc-4812-99c7-d4f56d0adeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:38.090151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379831e-23fc-4812-99c7-d4f56d0adeae admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:38.090367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379831e-23fc-4812-99c7-d4f56d0adeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:38.090813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379831e-23fc-4812-99c7-d4f56d0adeae admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:38.091741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379831e-23fc-4812-99c7-d4f56d0adeae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:38.093621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379831e-23fc-4812-99c7-d4f56d0adeae admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:38.141389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 821/1653] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:38.145298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ac0fe-7b42-4c53-b5cb-8d5d9643ffcf admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:38.188748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ac0fe-7b42-4c53-b5cb-8d5d9643ffcf admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:38.215136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:38.215834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:38.216115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:38.216390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] PATH_INFO: `/v3/projects/284ebd2544ae4c6d8e4ff13a4330c3f8/users/235894f779884573b8263c00954bf62d/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:38.233816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] RBAC: Authorizing `identity:create_grant(project_id=284ebd2544ae4c6d8e4ff13a4330c3f8, user_id=235894f779884573b8263c00954bf62d, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:38.245539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:38.304418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 833/1654] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:38 2022] PUT /identity/v3/projects/284ebd2544ae4c6d8e4ff13a4330c3f8/users/235894f779884573b8263c00954bf62d/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:17:38.318689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbeb98-82cf-4e9b-951e-f06f5ce7058f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:38.318689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbeb98-82cf-4e9b-951e-f06f5ce7058f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:38.318689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbeb98-82cf-4e9b-951e-f06f5ce7058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:38.444133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-88dbeb98-82cf-4e9b-951e-f06f5ce7058f None None] MFA Rules not processed for user `235894f779884573b8263c00954bf62d`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:38.446137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-88dbeb98-82cf-4e9b-951e-f06f5ce7058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:38.576071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 822/1655] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:38.591351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:38.593730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:38.735312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e51e5-6cd7-4af4-b59b-72071562f94d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:38.746224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:38.750122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:38.751709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:38.752006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:38.782816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:38.785900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:38.827627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 834/1656] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:39.031105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbeb98-82cf-4e9b-951e-f06f5ce7058f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:39.051378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbeb98-82cf-4e9b-951e-f06f5ce7058f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:39.061696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:39.062710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:39.063220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:39.063700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:39.245472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:39.246832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:39.267373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 823/1657] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:42.203184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:42.203184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:42.253486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:42.255276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:42.430430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe05f72-cbbb-4fba-9ab4-1ad501d10dcf service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:42.444853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:42.452116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:42.452468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:42.452784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] PATH_INFO: `/v3/users/50cc345e23a44acfb5dcf189a56124b1` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:42.456591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] RBAC: Authorizing `identity:delete_user(user_id=50cc345e23a44acfb5dcf189a56124b1)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:42.460299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de68e5f-8504-4590-bcf3-a89391a642ac service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:42.461951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:42.473782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:42.479058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:42.479607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:42.480360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] PATH_INFO: `/v3/users/02363f22cad04443a08d7fc03f8be3db` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:42.484407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02363f22cad04443a08d7fc03f8be3db)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:42.495756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:42.545883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:42.577377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:42.598218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50cc345e23a44acfb5dcf189a56124b1'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.629260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50cc345e23a44acfb5dcf189a56124b1'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.630296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50cc345e23a44acfb5dcf189a56124b1'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.637780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02363f22cad04443a08d7fc03f8be3db'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.677309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50cc345e23a44acfb5dcf189a56124b1'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.678821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4cdf6547-ef18-4c4a-926f-9ac84ae705be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50cc345e23a44acfb5dcf189a56124b1'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.707655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02363f22cad04443a08d7fc03f8be3db'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.709668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02363f22cad04443a08d7fc03f8be3db'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.711577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02363f22cad04443a08d7fc03f8be3db'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.716886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 835/1658] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:42 2022] DELETE /identity/v3/users/50cc345e23a44acfb5dcf189a56124b1 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:42.722871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76c68-4d5f-4292-84ed-227e98fee490 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:42.723233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76c68-4d5f-4292-84ed-227e98fee490 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:42.723450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76c68-4d5f-4292-84ed-227e98fee490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:42.742532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fcf311ae-bb8a-4a00-b07f-06847f35fa00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02363f22cad04443a08d7fc03f8be3db'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:42.772468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 824/1659] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:42 2022] DELETE /identity/v3/users/02363f22cad04443a08d7fc03f8be3db => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:42.777085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5b9a-c47c-4e99-91cc-ddb911c15007 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:42.778033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5b9a-c47c-4e99-91cc-ddb911c15007 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:42.778033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5b9a-c47c-4e99-91cc-ddb911c15007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:42.800944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-92b76c68-4d5f-4292-84ed-227e98fee490 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:42.803499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-92b76c68-4d5f-4292-84ed-227e98fee490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:42.846160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a6bd5b9a-c47c-4e99-91cc-ddb911c15007 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:42.848768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a6bd5b9a-c47c-4e99-91cc-ddb911c15007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:43.020335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 836/1660] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:43.070523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 825/1661] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:43.078394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b76c68-4d5f-4292-84ed-227e98fee490 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:43.080116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-92b76c68-4d5f-4292-84ed-227e98fee490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:43.089411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd5b9a-c47c-4e99-91cc-ddb911c15007 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:43.092090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a6bd5b9a-c47c-4e99-91cc-ddb911c15007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:43.259606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b76c68-4d5f-4292-84ed-227e98fee490 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:43.274547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd5b9a-c47c-4e99-91cc-ddb911c15007 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:43.275910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ef498-445d-479d-9bdc-84cc96d21dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:43.276551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ef498-445d-479d-9bdc-84cc96d21dcf service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:43.276856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ef498-445d-479d-9bdc-84cc96d21dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:43.277129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ef498-445d-479d-9bdc-84cc96d21dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:43.289290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:43.293074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:43.294577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:43.294882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:43.319140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ef498-445d-479d-9bdc-84cc96d21dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:43.320695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ef498-445d-479d-9bdc-84cc96d21dcf service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:43.329614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:43.331133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:43.355694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 837/1662] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:43.385808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 826/1663] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:44.061004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182a2c-985e-4db9-b6f0-cc9bb69855d9 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:44.061283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182a2c-985e-4db9-b6f0-cc9bb69855d9 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:44.061561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6182a2c-985e-4db9-b6f0-cc9bb69855d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:44.153609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b6182a2c-985e-4db9-b6f0-cc9bb69855d9 None None] MFA Rules not processed for user `25c3f0a2e1fd4067b1804ef42faa46d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:44.154893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b6182a2c-985e-4db9-b6f0-cc9bb69855d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:44.248397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:44.300100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 838/1664] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:44.315255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:44.317461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:44.491824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:44.635606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc0b23-4e54-4331-b618-f853f483a48b service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:44.650575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:44.651225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:44.651509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:44.651793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:44.680973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:44.685317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:44.717555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 827/1665] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:45.359920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6182a2c-985e-4db9-b6f0-cc9bb69855d9 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:45.362897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b6182a2c-985e-4db9-b6f0-cc9bb69855d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:45.416230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6182a2c-985e-4db9-b6f0-cc9bb69855d9 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:45.435204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:45.441797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:45.442095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:45.442371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] PATH_INFO: `/v3/projects/e5d6fc02c5ab40cba5d60a67717786e5` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:45.447839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d6fc02c5ab40cba5d60a67717786e5)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:45.456174 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:45.589230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d6fc02c5ab40cba5d60a67717786e5', 'request_id': 'req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:45.617305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d6fc02c5ab40cba5d60a67717786e5', 'request_id': 'req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:45.635862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d6fc02c5ab40cba5d60a67717786e5', 'request_id': 'req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:45.655970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:45.658908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:45.681175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] The token cache is being invalidate because project e5d6fc02c5ab40cba5d60a67717786e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:17:45.684212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:45.686140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-dfe850ef-d8bc-4d1e-8d60-dea9f1280dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:45.695228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d7b88-da05-46b9-933b-f5781aaea775 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:45.697428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 839/1666] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:17:45 2022] DELETE /identity/v3/projects/e5d6fc02c5ab40cba5d60a67717786e5 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:45.701414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d36504-2cda-46ad-87a9-81b02e5475a8 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:45.701956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d36504-2cda-46ad-87a9-81b02e5475a8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:45.702396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d36504-2cda-46ad-87a9-81b02e5475a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:45.767838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-29d36504-2cda-46ad-87a9-81b02e5475a8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:45.776360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-29d36504-2cda-46ad-87a9-81b02e5475a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:45.925831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 840/1667] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:45.934118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c1561-0243-4725-aed7-54e702972df8 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:45.935604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c1561-0243-4725-aed7-54e702972df8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:45.935983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c1561-0243-4725-aed7-54e702972df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:45.951873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:45.953277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:45.954682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:45.955071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] PATH_INFO: `/v3/projects/1ecdbc1a962e47a2b1885519551b23e6` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:45.958194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ecdbc1a962e47a2b1885519551b23e6)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:45.963171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.001538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b36c1561-0243-4725-aed7-54e702972df8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:46.004368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b36c1561-0243-4725-aed7-54e702972df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:46.030589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 841/1668] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:46.031764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c1561-0243-4725-aed7-54e702972df8 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:46.074378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c1561-0243-4725-aed7-54e702972df8 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.125409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ecdbc1a962e47a2b1885519551b23e6', 'request_id': 'req-921780d1-36cd-480e-b2f5-ed5183d95117'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:46.153240 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ecdbc1a962e47a2b1885519551b23e6', 'request_id': 'req-921780d1-36cd-480e-b2f5-ed5183d95117'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:46.168359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ecdbc1a962e47a2b1885519551b23e6', 'request_id': 'req-921780d1-36cd-480e-b2f5-ed5183d95117'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:46.203231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] The token cache is being invalidate because project 1ecdbc1a962e47a2b1885519551b23e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:17:46.206633 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:46.207459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-921780d1-36cd-480e-b2f5-ed5183d95117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:46.215854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 828/1669] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:17:45 2022] DELETE /identity/v3/projects/1ecdbc1a962e47a2b1885519551b23e6 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:17:46.220766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381ccac-aed2-42d1-9122-a78d37724ec6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.222083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381ccac-aed2-42d1-9122-a78d37724ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.222468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381ccac-aed2-42d1-9122-a78d37724ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.278969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.284443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.284910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.285340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.287838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:46.289437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0381ccac-aed2-42d1-9122-a78d37724ec6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:46.292630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:46.293907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0381ccac-aed2-42d1-9122-a78d37724ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:46.300664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.307150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:46.327054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 829/1670] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:46.330733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 842/1671] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:46.332527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f65fb-5cdb-4a4e-b6fa-8c4319105aa6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.332970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f65fb-5cdb-4a4e-b6fa-8c4319105aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.333358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f65fb-5cdb-4a4e-b6fa-8c4319105aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.338801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:46.341445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:46.364755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867e6d5-d022-4ef6-82fe-ac2002db738b admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.374860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e18f65fb-5cdb-4a4e-b6fa-8c4319105aa6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:46.377058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e18f65fb-5cdb-4a4e-b6fa-8c4319105aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:46.380312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4f50-8207-4059-a353-9990b0332327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.381758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4f50-8207-4059-a353-9990b0332327 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.382494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4f50-8207-4059-a353-9990b0332327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.383351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d4f50-8207-4059-a353-9990b0332327 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.384597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4f50-8207-4059-a353-9990b0332327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:46.388516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d4f50-8207-4059-a353-9990b0332327 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.397673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 830/1672] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:46.402001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f65fb-5cdb-4a4e-b6fa-8c4319105aa6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:46.419493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f65fb-5cdb-4a4e-b6fa-8c4319105aa6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.430196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.431023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.431402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.431914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.432984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:46.433357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:46.442070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.447605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 843/1673] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:46 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:46.450505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4f50-8207-4059-a353-9990b0332327 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:46.467565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 831/1674] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:46.471112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:46.486469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc86cc-cdb0-4c58-af68-03b38d6e6087 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.535950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:46.647788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d4f50-8207-4059-a353-9990b0332327 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.659365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b94c35-5e3b-44ee-b006-a58ea8986e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.660147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b94c35-5e3b-44ee-b006-a58ea8986e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.660434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b94c35-5e3b-44ee-b006-a58ea8986e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.660736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b94c35-5e3b-44ee-b006-a58ea8986e03 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.661415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b94c35-5e3b-44ee-b006-a58ea8986e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:46.663423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b94c35-5e3b-44ee-b006-a58ea8986e03 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.674013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b813a28-6c06-4424-a2f7-e8667460e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.674675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813a28-6c06-4424-a2f7-e8667460e57d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.678154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813a28-6c06-4424-a2f7-e8667460e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.679758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b813a28-6c06-4424-a2f7-e8667460e57d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.680767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b813a28-6c06-4424-a2f7-e8667460e57d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:46.683130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b813a28-6c06-4424-a2f7-e8667460e57d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.735819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 844/1675] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:46 2022] POST /identity/v3/users => generated 302 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:46.740459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b94c35-5e3b-44ee-b006-a58ea8986e03 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:46.780319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 832/1676] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:46 2022] POST /identity/v3/projects => generated 381 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:46.780958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b94c35-5e3b-44ee-b006-a58ea8986e03 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.787007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b813a28-6c06-4424-a2f7-e8667460e57d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:46.939744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b813a28-6c06-4424-a2f7-e8667460e57d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.961384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2449e60-7c04-4702-bd8c-31db677c9df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.962023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2449e60-7c04-4702-bd8c-31db677c9df8 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.962303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2449e60-7c04-4702-bd8c-31db677c9df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.962570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2449e60-7c04-4702-bd8c-31db677c9df8 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.963218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2449e60-7c04-4702-bd8c-31db677c9df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:46.966770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2449e60-7c04-4702-bd8c-31db677c9df8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.975704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a7af-26cb-4800-8450-78b3dfb5f03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.978150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a7af-26cb-4800-8450-78b3dfb5f03b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.990590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a7af-26cb-4800-8450-78b3dfb5f03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.990930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a7af-26cb-4800-8450-78b3dfb5f03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.992008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787a7af-26cb-4800-8450-78b3dfb5f03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:47.004511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787a7af-26cb-4800-8450-78b3dfb5f03b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:47.032596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 833/1677] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:46 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:47.059322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2449e60-7c04-4702-bd8c-31db677c9df8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:47.069806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 845/1678] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:47.079493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a7af-26cb-4800-8450-78b3dfb5f03b admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:47.096854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2449e60-7c04-4702-bd8c-31db677c9df8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:47.112886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe6911-90f0-4b69-9dfd-684dde1aacfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:47.113517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6911-90f0-4b69-9dfd-684dde1aacfb admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.113800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6911-90f0-4b69-9dfd-684dde1aacfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.114068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6911-90f0-4b69-9dfd-684dde1aacfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.114857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6911-90f0-4b69-9dfd-684dde1aacfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:47.116768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6911-90f0-4b69-9dfd-684dde1aacfb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:47.118018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a7af-26cb-4800-8450-78b3dfb5f03b admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:47.128597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7da65-38c6-43d3-95ff-d401deafd0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:47.158117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 834/1679] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:47.159428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe6911-90f0-4b69-9dfd-684dde1aacfb admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:47.167991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7da65-38c6-43d3-95ff-d401deafd0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.168284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7da65-38c6-43d3-95ff-d401deafd0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.171275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7da65-38c6-43d3-95ff-d401deafd0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.172058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7da65-38c6-43d3-95ff-d401deafd0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:47.181655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7da65-38c6-43d3-95ff-d401deafd0b8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:47.184886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe6911-90f0-4b69-9dfd-684dde1aacfb admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:47.198676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72271f87-cda8-4352-bf52-52a9a4fda353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:47.199310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72271f87-cda8-4352-bf52-52a9a4fda353 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.199591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72271f87-cda8-4352-bf52-52a9a4fda353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.199864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72271f87-cda8-4352-bf52-52a9a4fda353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.200510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72271f87-cda8-4352-bf52-52a9a4fda353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:47.201569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 846/1680] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:47.202230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72271f87-cda8-4352-bf52-52a9a4fda353 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:47.202656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7da65-38c6-43d3-95ff-d401deafd0b8 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:47.218894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 835/1681] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:47.222902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72271f87-cda8-4352-bf52-52a9a4fda353 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:47.227345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7da65-38c6-43d3-95ff-d401deafd0b8 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:47.232629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e493d-7f71-41b1-8346-381221a5ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:47.235034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e493d-7f71-41b1-8346-381221a5ca71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.235511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e493d-7f71-41b1-8346-381221a5ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.235797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e493d-7f71-41b1-8346-381221a5ca71 admin admin] PATH_INFO: `/v3/projects/804deeee879641388c6ba50c0b2154d6/users/609362f63dd848aa94a41d3dca4fdd5e/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.237328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72271f87-cda8-4352-bf52-52a9a4fda353 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:47.251739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5bf98-5a8d-451e-87e6-06181ef2c7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:47.252931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5bf98-5a8d-451e-87e6-06181ef2c7b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.253217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5bf98-5a8d-451e-87e6-06181ef2c7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.253492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5bf98-5a8d-451e-87e6-06181ef2c7b5 admin admin] PATH_INFO: `/v3/projects/feabe9d5812e46c2ae8204821517493d/users/d4eaa129baf94a04afeefd230a91285a/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.274960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5bf98-5a8d-451e-87e6-06181ef2c7b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feabe9d5812e46c2ae8204821517493d, user_id=d4eaa129baf94a04afeefd230a91285a, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:47.279592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5bf98-5a8d-451e-87e6-06181ef2c7b5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:47.284409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e493d-7f71-41b1-8346-381221a5ca71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804deeee879641388c6ba50c0b2154d6, user_id=609362f63dd848aa94a41d3dca4fdd5e, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:47.290900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e493d-7f71-41b1-8346-381221a5ca71 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:47.335636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 836/1682] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:47 2022] PUT /identity/v3/projects/feabe9d5812e46c2ae8204821517493d/users/d4eaa129baf94a04afeefd230a91285a/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:17:47.338220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 847/1683] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:47 2022] PUT /identity/v3/projects/804deeee879641388c6ba50c0b2154d6/users/609362f63dd848aa94a41d3dca4fdd5e/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:47.340755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f481-9ea3-4ec3-a020-5861d01ce7a5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.341258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f481-9ea3-4ec3-a020-5861d01ce7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.341752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50f481-9ea3-4ec3-a020-5861d01ce7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.350788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2161e59-fa89-4e30-90b1-68f2093a2ae5 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.351234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2161e59-fa89-4e30-90b1-68f2093a2ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.351652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2161e59-fa89-4e30-90b1-68f2093a2ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.393905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7e50f481-9ea3-4ec3-a020-5861d01ce7a5 None None] MFA Rules not processed for user `d4eaa129baf94a04afeefd230a91285a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:47.395357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7e50f481-9ea3-4ec3-a020-5861d01ce7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:47.402294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c2161e59-fa89-4e30-90b1-68f2093a2ae5 None None] MFA Rules not processed for user `609362f63dd848aa94a41d3dca4fdd5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:47.404427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c2161e59-fa89-4e30-90b1-68f2093a2ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:47.427184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:47.467279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:47.535759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 837/1684] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:47.535759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa8ec7-f0d1-4749-a8f4-211bb5dfeb68 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.535759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa8ec7-f0d1-4749-a8f4-211bb5dfeb68 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.535759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa8ec7-f0d1-4749-a8f4-211bb5dfeb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.607422 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 848/1685] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:47 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:47.611450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866bd3-13f5-4237-9a63-e0283f91c7ce None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.611953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866bd3-13f5-4237-9a63-e0283f91c7ce None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.612550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866bd3-13f5-4237-9a63-e0283f91c7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.627452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-47fa8ec7-f0d1-4749-a8f4-211bb5dfeb68 None None] MFA Rules not processed for user `6e09978ed8b3490db5f92823768f1d15`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:47.629495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-47fa8ec7-f0d1-4749-a8f4-211bb5dfeb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:47.700252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-60866bd3-13f5-4237-9a63-e0283f91c7ce None None] MFA Rules not processed for user `a288c513afd94adaa9f53f9f65dc9dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:47.702404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-60866bd3-13f5-4237-9a63-e0283f91c7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:47.808468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 838/1686] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:47.809375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa8ec7-f0d1-4749-a8f4-211bb5dfeb68 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:47.900320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 849/1687] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:47.900320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60866bd3-13f5-4237-9a63-e0283f91c7ce None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:48.063419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa8ec7-f0d1-4749-a8f4-211bb5dfeb68 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:48.085570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:48.086336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.086725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.087145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.089665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:48.094051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:48.115085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60866bd3-13f5-4237-9a63-e0283f91c7ce None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:48.126427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:48.127701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.127701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.128682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.129462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:48.133449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:48.171747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 839/1688] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:47 2022] POST /identity/v3/projects => generated 381 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:48.171747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:48.174286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:48.212312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 850/1689] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:47 2022] POST /identity/v3/projects => generated 403 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:48.215789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:48.228741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:48.322583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:48.345392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:48.412541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4921a45-21a3-465e-a82c-59a6408d5647 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:48.436689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d610d-81a7-49dd-ac39-952d4f8552e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:48.437448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d610d-81a7-49dd-ac39-952d4f8552e3 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.437842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d610d-81a7-49dd-ac39-952d4f8552e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.438225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d610d-81a7-49dd-ac39-952d4f8552e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.452374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c72c7-aad3-4381-ac12-17f297013c2b admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:48.466919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f9ccb-dbba-46e6-9e72-0442f7b9f258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:48.467699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9ccb-dbba-46e6-9e72-0442f7b9f258 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.469799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9ccb-dbba-46e6-9e72-0442f7b9f258 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.470199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f9ccb-dbba-46e6-9e72-0442f7b9f258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.578290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f9ccb-dbba-46e6-9e72-0442f7b9f258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:48.580179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f9ccb-dbba-46e6-9e72-0442f7b9f258 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:48.593303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 851/1690] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:48.594280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f9ccb-dbba-46e6-9e72-0442f7b9f258 service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:48.602746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d610d-81a7-49dd-ac39-952d4f8552e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:48.607717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d610d-81a7-49dd-ac39-952d4f8552e3 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:48.635513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 840/1691] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:48.639833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d610d-81a7-49dd-ac39-952d4f8552e3 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:48.795526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f9ccb-dbba-46e6-9e72-0442f7b9f258 service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:48.813091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c5bbf-5d81-40ed-9dc3-1e41aac2cc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:48.814066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c5bbf-5d81-40ed-9dc3-1e41aac2cc7b admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.814502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c5bbf-5d81-40ed-9dc3-1e41aac2cc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.814908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c5bbf-5d81-40ed-9dc3-1e41aac2cc7b admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.815845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c5bbf-5d81-40ed-9dc3-1e41aac2cc7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:48.820029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c5bbf-5d81-40ed-9dc3-1e41aac2cc7b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:48.850814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d610d-81a7-49dd-ac39-952d4f8552e3 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:48.877439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1976ef4-16d7-433a-ab3e-495d5caf2e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:48.881692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976ef4-16d7-433a-ab3e-495d5caf2e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.881896 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976ef4-16d7-433a-ab3e-495d5caf2e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.882459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976ef4-16d7-433a-ab3e-495d5caf2e6f admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.891553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1976ef4-16d7-433a-ab3e-495d5caf2e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:48.894757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 852/1692] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:48 2022] POST /identity/v3/users => generated 297 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:48.899884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1976ef4-16d7-433a-ab3e-495d5caf2e6f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:48.903093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29016a3a-ff88-40d9-a0fe-7239e6a48a92 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:48.906532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29016a3a-ff88-40d9-a0fe-7239e6a48a92 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:48.906907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29016a3a-ff88-40d9-a0fe-7239e6a48a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:48.994710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 841/1693] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:48 2022] POST /identity/v3/users => generated 308 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:49.007501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1976ef4-16d7-433a-ab3e-495d5caf2e6f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:49.054604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-29016a3a-ff88-40d9-a0fe-7239e6a48a92 None None] MFA Rules not processed for user `97c9201758884fdbb080f85337a5ce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:49.058027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-29016a3a-ff88-40d9-a0fe-7239e6a48a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:49.059995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1976ef4-16d7-433a-ab3e-495d5caf2e6f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.081706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20189fa6-d595-4eaa-9043-2d225af3235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.088461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20189fa6-d595-4eaa-9043-2d225af3235b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.088461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20189fa6-d595-4eaa-9043-2d225af3235b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.088461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20189fa6-d595-4eaa-9043-2d225af3235b admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.089665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20189fa6-d595-4eaa-9043-2d225af3235b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:49.109410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20189fa6-d595-4eaa-9043-2d225af3235b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:49.159000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 842/1694] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:49.168808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20189fa6-d595-4eaa-9043-2d225af3235b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:49.198851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20189fa6-d595-4eaa-9043-2d225af3235b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.231514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82f960-58f0-48bc-b299-89364f74b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.235162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82f960-58f0-48bc-b299-89364f74b488 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.238391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82f960-58f0-48bc-b299-89364f74b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.238391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82f960-58f0-48bc-b299-89364f74b488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.238391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82f960-58f0-48bc-b299-89364f74b488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:49.246185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82f960-58f0-48bc-b299-89364f74b488 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:49.278082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:49.294307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 843/1695] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:49.294307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82f960-58f0-48bc-b299-89364f74b488 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:49.321234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82f960-58f0-48bc-b299-89364f74b488 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.334405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca748c0b-c245-4f1b-9ca5-8ccb3dee2085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.335192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca748c0b-c245-4f1b-9ca5-8ccb3dee2085 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.335473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca748c0b-c245-4f1b-9ca5-8ccb3dee2085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.337910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca748c0b-c245-4f1b-9ca5-8ccb3dee2085 admin admin] PATH_INFO: `/v3/projects/d498115951cd4bd5bb4bee9da2d8d123/users/7331e45a2e4c493f91c3b3e653d67696/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.380197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 853/1696] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:49.380197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29016a3a-ff88-40d9-a0fe-7239e6a48a92 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:49.402980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca748c0b-c245-4f1b-9ca5-8ccb3dee2085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d498115951cd4bd5bb4bee9da2d8d123, user_id=7331e45a2e4c493f91c3b3e653d67696, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:49.404457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29016a3a-ff88-40d9-a0fe-7239e6a48a92 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.405693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca748c0b-c245-4f1b-9ca5-8ccb3dee2085 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:49.415538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc1fc5-3f8e-4ddd-93e5-e5d7adcffce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.416373 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc1fc5-3f8e-4ddd-93e5-e5d7adcffce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.416798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc1fc5-3f8e-4ddd-93e5-e5d7adcffce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.417185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc1fc5-3f8e-4ddd-93e5-e5d7adcffce0 admin admin] PATH_INFO: `/v3/projects/0c7193a191e74360839b516e36440213/users/d4e3836e096f4fc790b503655f1638e0/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.468047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc1fc5-3f8e-4ddd-93e5-e5d7adcffce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7193a191e74360839b516e36440213, user_id=d4e3836e096f4fc790b503655f1638e0, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:49.469573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 844/1697] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:49 2022] PUT /identity/v3/projects/d498115951cd4bd5bb4bee9da2d8d123/users/7331e45a2e4c493f91c3b3e653d67696/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:49.473405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2cb6d-27df-44c7-9dd5-fb98d51c9ea1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.473883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2cb6d-27df-44c7-9dd5-fb98d51c9ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.475318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2cb6d-27df-44c7-9dd5-fb98d51c9ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.486700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc1fc5-3f8e-4ddd-93e5-e5d7adcffce0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:49.517262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 854/1698] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:49 2022] PUT /identity/v3/projects/0c7193a191e74360839b516e36440213/users/d4e3836e096f4fc790b503655f1638e0/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:49.517592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc1fc5-3f8e-4ddd-93e5-e5d7adcffce0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:49.528169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dfc2cb6d-27df-44c7-9dd5-fb98d51c9ea1 None None] MFA Rules not processed for user `97c9201758884fdbb080f85337a5ce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:49.531995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dfc2cb6d-27df-44c7-9dd5-fb98d51c9ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:49.612358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:49.640471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc1fc5-3f8e-4ddd-93e5-e5d7adcffce0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.654228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc0718-0f4e-4bfe-9478-86460c26f100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.655333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0718-0f4e-4bfe-9478-86460c26f100 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.657333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0718-0f4e-4bfe-9478-86460c26f100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.657831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0718-0f4e-4bfe-9478-86460c26f100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.659073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc0718-0f4e-4bfe-9478-86460c26f100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:49.664498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc0718-0f4e-4bfe-9478-86460c26f100 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:49.689505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 845/1699] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:49.692431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 855/1700] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:49.696894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2cb6d-27df-44c7-9dd5-fb98d51c9ea1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:49.697671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc0718-0f4e-4bfe-9478-86460c26f100 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:49.720973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc0718-0f4e-4bfe-9478-86460c26f100 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.729352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7af21-c1a5-43a8-b9b4-0c6421f33748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.730505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7af21-c1a5-43a8-b9b4-0c6421f33748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.730990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7af21-c1a5-43a8-b9b4-0c6421f33748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.731421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7af21-c1a5-43a8-b9b4-0c6421f33748 admin admin] PATH_INFO: `/v3/projects/d498115951cd4bd5bb4bee9da2d8d123/users/7331e45a2e4c493f91c3b3e653d67696/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.734429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7af21-c1a5-43a8-b9b4-0c6421f33748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d498115951cd4bd5bb4bee9da2d8d123, user_id=7331e45a2e4c493f91c3b3e653d67696, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:49.735270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2cb6d-27df-44c7-9dd5-fb98d51c9ea1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.739243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7af21-c1a5-43a8-b9b4-0c6421f33748 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:49.752410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72885b-e078-4204-ac48-320fcbf13509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.753216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72885b-e078-4204-ac48-320fcbf13509 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.753553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72885b-e078-4204-ac48-320fcbf13509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.760040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 856/1701] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:49 2022] PUT /identity/v3/projects/d498115951cd4bd5bb4bee9da2d8d123/users/7331e45a2e4c493f91c3b3e653d67696/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:17:49.760929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72885b-e078-4204-ac48-320fcbf13509 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.761396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7af21-c1a5-43a8-b9b4-0c6421f33748 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:49.761643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72885b-e078-4204-ac48-320fcbf13509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:49.767690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72885b-e078-4204-ac48-320fcbf13509 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:49.805962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 846/1702] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:49.807565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff08993-2afb-4c39-9847-26252fbc19f1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.807869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff08993-2afb-4c39-9847-26252fbc19f1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.808129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff08993-2afb-4c39-9847-26252fbc19f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:49.892462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cff08993-2afb-4c39-9847-26252fbc19f1 None None] MFA Rules not processed for user `7331e45a2e4c493f91c3b3e653d67696`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:49.895956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cff08993-2afb-4c39-9847-26252fbc19f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:49.965548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7af21-c1a5-43a8-b9b4-0c6421f33748 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:49.984079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:49.986053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:49.987354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:49.987799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:50.102498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:50.141835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:50.143684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:50.147693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 847/1703] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:50.148604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff08993-2afb-4c39-9847-26252fbc19f1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:50.163121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 857/1704] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:50.164131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:50.165831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:50.292110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5d8a2-86a7-4026-a3b8-2e400a5549ff service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:50.297557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80e7b0-d406-49e6-9e81-d12fba30eea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:50.298281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80e7b0-d406-49e6-9e81-d12fba30eea2 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:50.298661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80e7b0-d406-49e6-9e81-d12fba30eea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:50.299031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80e7b0-d406-49e6-9e81-d12fba30eea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:50.317206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff08993-2afb-4c39-9847-26252fbc19f1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:50.324992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d592006-3c6e-4e00-b558-157f48cec136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:50.325875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592006-3c6e-4e00-b558-157f48cec136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:50.326322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592006-3c6e-4e00-b558-157f48cec136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:50.326758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592006-3c6e-4e00-b558-157f48cec136 admin admin] PATH_INFO: `/v3/projects/0c7193a191e74360839b516e36440213/users/d4e3836e096f4fc790b503655f1638e0/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:50.330961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d592006-3c6e-4e00-b558-157f48cec136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7193a191e74360839b516e36440213, user_id=d4e3836e096f4fc790b503655f1638e0, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:50.334637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d592006-3c6e-4e00-b558-157f48cec136 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:50.345231 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80e7b0-d406-49e6-9e81-d12fba30eea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:50.346880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80e7b0-d406-49e6-9e81-d12fba30eea2 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:50.363782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 848/1705] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:50 2022] PUT /identity/v3/projects/0c7193a191e74360839b516e36440213/users/d4e3836e096f4fc790b503655f1638e0/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:50.365646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe13710-cbc7-457b-afad-c68f522432f6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:50.366314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe13710-cbc7-457b-afad-c68f522432f6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:50.366588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe13710-cbc7-457b-afad-c68f522432f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:50.376634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 858/1706] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:50.448875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dfe13710-cbc7-457b-afad-c68f522432f6 None None] MFA Rules not processed for user `d4e3836e096f4fc790b503655f1638e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:50.450164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dfe13710-cbc7-457b-afad-c68f522432f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:50.476490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fb2e3-5af5-445d-a560-411819c24913 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:50.477056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fb2e3-5af5-445d-a560-411819c24913 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:50.477341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fb2e3-5af5-445d-a560-411819c24913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:50.531777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-222fb2e3-5af5-445d-a560-411819c24913 None None] MFA Rules not processed for user `fd0f7f51482040d7a89df3f7502c2638`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:50.542717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-222fb2e3-5af5-445d-a560-411819c24913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:50.561057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:50.641067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 849/1707] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:50.642220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe13710-cbc7-457b-afad-c68f522432f6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:50.738373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe13710-cbc7-457b-afad-c68f522432f6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:50.744256 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3886ecf-c87f-424e-b4dd-9cd015fda8a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:50.745099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3886ecf-c87f-424e-b4dd-9cd015fda8a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:50.745460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3886ecf-c87f-424e-b4dd-9cd015fda8a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:50.745739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3886ecf-c87f-424e-b4dd-9cd015fda8a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:50.770871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 859/1708] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:50.781392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb494d7-95db-4666-9e4d-e960c2c677f3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:50.781787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb494d7-95db-4666-9e4d-e960c2c677f3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:50.782169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb494d7-95db-4666-9e4d-e960c2c677f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:50.837659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-9eb494d7-95db-4666-9e4d-e960c2c677f3 None None] MFA Rules not processed for user `fd0f7f51482040d7a89df3f7502c2638`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:50.839006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9eb494d7-95db-4666-9e4d-e960c2c677f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:50.859581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3886ecf-c87f-424e-b4dd-9cd015fda8a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:50.861023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3886ecf-c87f-424e-b4dd-9cd015fda8a8 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:50.877600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 860/1709] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:50.878718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb494d7-95db-4666-9e4d-e960c2c677f3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:50.884995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9eb494d7-95db-4666-9e4d-e960c2c677f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:50.890403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 850/1710] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:50.891568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3886ecf-c87f-424e-b4dd-9cd015fda8a8 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:50.924481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb494d7-95db-4666-9e4d-e960c2c677f3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:50.949128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d187f71-764f-4fcd-b6e4-ddd1c7a90a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:50.949941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187f71-764f-4fcd-b6e4-ddd1c7a90a79 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:50.950336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187f71-764f-4fcd-b6e4-ddd1c7a90a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:50.950729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187f71-764f-4fcd-b6e4-ddd1c7a90a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:50.977713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d187f71-764f-4fcd-b6e4-ddd1c7a90a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:50.979469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d187f71-764f-4fcd-b6e4-ddd1c7a90a79 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:50.997439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 861/1711] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:50 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:51.073275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e340-1100-4671-a952-4ea0cbf5a8ee None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:51.073668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e340-1100-4671-a952-4ea0cbf5a8ee None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:51.074049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e340-1100-4671-a952-4ea0cbf5a8ee None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:51.075748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 862/1712] 10.222.0.95 () {54 vars in 912 bytes} [Wed Aug 31 19:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:17:51.098273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caf394-4e00-40de-8528-4d744076ffb1 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:51.098621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caf394-4e00-40de-8528-4d744076ffb1 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:51.098966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caf394-4e00-40de-8528-4d744076ffb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:51.110158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3886ecf-c87f-424e-b4dd-9cd015fda8a8 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:51.151914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:51.155592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:51.156660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:51.158106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:51.190763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:51.198629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:51.217860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-68caf394-4e00-40de-8528-4d744076ffb1 None None] MFA Rules not processed for user `5f1d5c29df30465ead9834b63c3e93bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:51.219112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-68caf394-4e00-40de-8528-4d744076ffb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:51.231908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 851/1713] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:51.307641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:51.348175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 863/1714] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:17:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:51.431012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:51.433153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:51.480021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29f4f9-ae7b-4ccf-b907-4c23b22a4996 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:51.500157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9115bca-0a3c-449b-8e7d-640fc3b5fdc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:51.500945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9115bca-0a3c-449b-8e7d-640fc3b5fdc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:51.503488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9115bca-0a3c-449b-8e7d-640fc3b5fdc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:51.503788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9115bca-0a3c-449b-8e7d-640fc3b5fdc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:51.546153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9115bca-0a3c-449b-8e7d-640fc3b5fdc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:51.579709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9115bca-0a3c-449b-8e7d-640fc3b5fdc8 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:51.628560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 852/1715] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:51.744122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a8a2-f80d-4350-b806-e9a4090f4279 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:51.744889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a8a2-f80d-4350-b806-e9a4090f4279 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:51.744889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a8a2-f80d-4350-b806-e9a4090f4279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:51.783928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-dc55a8a2-f80d-4350-b806-e9a4090f4279 None None] MFA Rules not processed for user `25c3f0a2e1fd4067b1804ef42faa46d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:51.790125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-dc55a8a2-f80d-4350-b806-e9a4090f4279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:51.836848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9115bca-0a3c-449b-8e7d-640fc3b5fdc8 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:51.842301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:51.981401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 864/1716] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:52.031963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9115bca-0a3c-449b-8e7d-640fc3b5fdc8 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:52.042735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:52.043402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:52.043680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:52.043951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] PATH_INFO: `/v3/users/609362f63dd848aa94a41d3dca4fdd5e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:52.045155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=609362f63dd848aa94a41d3dca4fdd5e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:52.047112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:52.151391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609362f63dd848aa94a41d3dca4fdd5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:52.181915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609362f63dd848aa94a41d3dca4fdd5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:52.186753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609362f63dd848aa94a41d3dca4fdd5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:52.188103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609362f63dd848aa94a41d3dca4fdd5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:52.200836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609362f63dd848aa94a41d3dca4fdd5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:52.229263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 853/1717] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:51 2022] DELETE /identity/v3/users/609362f63dd848aa94a41d3dca4fdd5e => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:52.231261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8fa3-d137-4767-94c1-0790f080fd5d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:52.232950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8fa3-d137-4767-94c1-0790f080fd5d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:52.233340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8fa3-d137-4767-94c1-0790f080fd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:52.274212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-398e8fa3-d137-4767-94c1-0790f080fd5d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:52.277675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-398e8fa3-d137-4767-94c1-0790f080fd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:52.444244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 865/1718] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:52.454699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:52.456877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:52.610783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338e6db-340f-4d40-9469-c8dfaac64586 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:52.617058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b42e2-234c-4179-adaa-cb39c017d825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:52.617864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b42e2-234c-4179-adaa-cb39c017d825 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:52.618313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b42e2-234c-4179-adaa-cb39c017d825 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:52.618645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b42e2-234c-4179-adaa-cb39c017d825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:52.634111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b42e2-234c-4179-adaa-cb39c017d825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:52.635593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b42e2-234c-4179-adaa-cb39c017d825 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:52.651723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 854/1719] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:52.981181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e60b-4f27-41ba-a84e-dd42531dd0fe None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:52.981970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e60b-4f27-41ba-a84e-dd42531dd0fe None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:52.982614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e60b-4f27-41ba-a84e-dd42531dd0fe None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:52.984526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 866/1720] 10.222.0.95 () {54 vars in 912 bytes} [Wed Aug 31 19:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:17:52.999767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4c70a-0013-45d9-83f7-e0333be455aa None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:53.000057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4c70a-0013-45d9-83f7-e0333be455aa None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:53.000335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4c70a-0013-45d9-83f7-e0333be455aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:53.057834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-81d4c70a-0013-45d9-83f7-e0333be455aa None None] MFA Rules not processed for user `5f1d5c29df30465ead9834b63c3e93bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:53.059352 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-81d4c70a-0013-45d9-83f7-e0333be455aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:53.103717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 855/1721] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:17:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:53.181200 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880e60b-4f27-41ba-a84e-dd42531dd0fe None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:53.183152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2880e60b-4f27-41ba-a84e-dd42531dd0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:53.225504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880e60b-4f27-41ba-a84e-dd42531dd0fe None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:53.239428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:53.241438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:53.241870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:53.242108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] PATH_INFO: `/v3/users/af37589186124cff9cd005e3814c09dd` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:53.243829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af37589186124cff9cd005e3814c09dd)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:53.246046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:53.313731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4c70a-0013-45d9-83f7-e0333be455aa None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:53.335267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4c70a-0013-45d9-83f7-e0333be455aa None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:53.357535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16471fbc-46cb-4867-ae18-fd09d482fa3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:53.358158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471fbc-46cb-4867-ae18-fd09d482fa3a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:53.358448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471fbc-46cb-4867-ae18-fd09d482fa3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:53.358762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471fbc-46cb-4867-ae18-fd09d482fa3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:53.378496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af37589186124cff9cd005e3814c09dd'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:53.382506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16471fbc-46cb-4867-ae18-fd09d482fa3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:53.383911 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16471fbc-46cb-4867-ae18-fd09d482fa3a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:53.403448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 856/1722] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:17:53.410030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af37589186124cff9cd005e3814c09dd'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:53.412881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af37589186124cff9cd005e3814c09dd'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:53.456057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af37589186124cff9cd005e3814c09dd'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:53.466477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f63f77c2-da88-40b6-97c5-7f206335e3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af37589186124cff9cd005e3814c09dd'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:53.499566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 867/1723] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:53 2022] DELETE /identity/v3/users/af37589186124cff9cd005e3814c09dd => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:53.506006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270de643-a42d-4496-84f5-e325304427d8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:53.506300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270de643-a42d-4496-84f5-e325304427d8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:53.506577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270de643-a42d-4496-84f5-e325304427d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:53.527681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:53.574924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-270de643-a42d-4496-84f5-e325304427d8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:53.577953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-270de643-a42d-4496-84f5-e325304427d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:53.578676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16198ec4-a480-4b68-9cd4-bd05003edfec None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:53.579134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16198ec4-a480-4b68-9cd4-bd05003edfec None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:53.580323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16198ec4-a480-4b68-9cd4-bd05003edfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:53.602166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:53.614323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-16198ec4-a480-4b68-9cd4-bd05003edfec None None] MFA Rules not processed for user `6e09978ed8b3490db5f92823768f1d15`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:53.619989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-16198ec4-a480-4b68-9cd4-bd05003edfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:53.778201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 857/1724] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:53.795763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270de643-a42d-4496-84f5-e325304427d8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:53.797707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-270de643-a42d-4496-84f5-e325304427d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:53.839515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 868/1725] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:53.916297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270de643-a42d-4496-84f5-e325304427d8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:53.923327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:53.924410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:53.924841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:53.925312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:53.955273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:53.957841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:53.986290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 858/1726] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:17:54.220530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16198ec4-a480-4b68-9cd4-bd05003edfec None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:54.222963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-16198ec4-a480-4b68-9cd4-bd05003edfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:54.310819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:54.312987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:54.345089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e91b43-45cb-4918-a0f6-e514edd7f613 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:54.364085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:54.366840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:54.367127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:54.367442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] PATH_INFO: `/v3/projects/804deeee879641388c6ba50c0b2154d6` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:54.376981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=804deeee879641388c6ba50c0b2154d6)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:54.383106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:54.409955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16198ec4-a480-4b68-9cd4-bd05003edfec None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:54.420394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:54.421147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:54.421437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:54.422596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:54.426294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:54.458190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:54.461503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:54.496399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 869/1727] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:54.514558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804deeee879641388c6ba50c0b2154d6', 'request_id': 'req-1bb14bca-1a45-40a5-8733-39f86b9cde0d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.527158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:54.528883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:54.538936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804deeee879641388c6ba50c0b2154d6', 'request_id': 'req-1bb14bca-1a45-40a5-8733-39f86b9cde0d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.557882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804deeee879641388c6ba50c0b2154d6', 'request_id': 'req-1bb14bca-1a45-40a5-8733-39f86b9cde0d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.561895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925173dc-72ef-4180-826b-7eddbd4e613b service placement] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:54.602736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] The token cache is being invalidate because project 804deeee879641388c6ba50c0b2154d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:17:54.603062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.605209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.613171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 859/1728] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:17:54 2022] DELETE /identity/v3/projects/804deeee879641388c6ba50c0b2154d6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:54.617476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:54.621813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:54.696544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:54.697628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:54.698122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:54.698604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] PATH_INFO: `/v3/users/d4eaa129baf94a04afeefd230a91285a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:54.700369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4eaa129baf94a04afeefd230a91285a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:54.706197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:54.824471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:54.841331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:54.850343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4eaa129baf94a04afeefd230a91285a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.885851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4eaa129baf94a04afeefd230a91285a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.887493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4eaa129baf94a04afeefd230a91285a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.931311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4eaa129baf94a04afeefd230a91285a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.932906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5d519d4d-69f2-46a0-8e04-ea107bb1fd28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4eaa129baf94a04afeefd230a91285a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:54.982469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14bca-1a45-40a5-8733-39f86b9cde0d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:54.988927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:54.990413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:54.990567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:54.990894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] PATH_INFO: `/v3/users/d4e3836e096f4fc790b503655f1638e0` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:54.991826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 870/1729] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:54 2022] DELETE /identity/v3/users/d4eaa129baf94a04afeefd230a91285a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:54.992154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e3836e096f4fc790b503655f1638e0)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:54.995058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:54.996701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234113-6ce6-46c4-9ff4-ba6bd817480e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:54.998142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234113-6ce6-46c4-9ff4-ba6bd817480e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:55.001852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234113-6ce6-46c4-9ff4-ba6bd817480e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:55.047746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-be234113-6ce6-46c4-9ff4-ba6bd817480e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:55.050599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-be234113-6ce6-46c4-9ff4-ba6bd817480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:55.074334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e3836e096f4fc790b503655f1638e0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.099258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e3836e096f4fc790b503655f1638e0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.100292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e3836e096f4fc790b503655f1638e0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.100858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e3836e096f4fc790b503655f1638e0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.126368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f7ef7363-747f-4baa-b183-13e2c67a6320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e3836e096f4fc790b503655f1638e0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.162129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 860/1730] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:54 2022] DELETE /identity/v3/users/d4e3836e096f4fc790b503655f1638e0 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:55.187895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a63be5-2a19-41d2-997e-c05b73d49df9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:55.188194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a63be5-2a19-41d2-997e-c05b73d49df9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:55.188470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a63be5-2a19-41d2-997e-c05b73d49df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:55.227714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-70a63be5-2a19-41d2-997e-c05b73d49df9 None None] MFA Rules not processed for user `a288c513afd94adaa9f53f9f65dc9dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:55.228997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-70a63be5-2a19-41d2-997e-c05b73d49df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:55.295074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 871/1731] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:55.296963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be234113-6ce6-46c4-9ff4-ba6bd817480e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:55.300225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-be234113-6ce6-46c4-9ff4-ba6bd817480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:55.346292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be234113-6ce6-46c4-9ff4-ba6bd817480e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:55.438610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 861/1732] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:55.440136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a63be5-2a19-41d2-997e-c05b73d49df9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:55.443996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-70a63be5-2a19-41d2-997e-c05b73d49df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:55.530920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:55.531862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:55.532282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:55.532726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] PATH_INFO: `/v3/projects/eafec54acec540ad9c9ed393e61b8707` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:55.535282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eafec54acec540ad9c9ed393e61b8707)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:55.540066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:55.684267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a63be5-2a19-41d2-997e-c05b73d49df9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:55.700291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:55.703449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eafec54acec540ad9c9ed393e61b8707', 'request_id': 'req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.731502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eafec54acec540ad9c9ed393e61b8707', 'request_id': 'req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.752433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eafec54acec540ad9c9ed393e61b8707', 'request_id': 'req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.756697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:55.797751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] The token cache is being invalidate because project eafec54acec540ad9c9ed393e61b8707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:17:55.798201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.805027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:55.807576 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 872/1733] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:17:55 2022] DELETE /identity/v3/projects/eafec54acec540ad9c9ed393e61b8707 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:55.811816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:55.815260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:55.855230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:55.855867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:55.856152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:55.856426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:55.858579 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:56.012481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d2428-3cc6-42f7-ba2e-22e61f152f6d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:56.025787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:56.026763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:56.027308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:56.027555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:56.027907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:56.028226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] PATH_INFO: `/v3/users/235894f779884573b8263c00954bf62d` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:56.029610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235894f779884573b8263c00954bf62d)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:56.031919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:56.056098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 862/1734] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:56.058036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:56.065773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:56.099094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b056c51-d582-4bc8-9580-8e5be12cafe5 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:56.111202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:56.111872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:56.112131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:56.112410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] PATH_INFO: `/v3/projects/0c7193a191e74360839b516e36440213` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:56.114165 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c7193a191e74360839b516e36440213)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:56.116756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:56.233353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235894f779884573b8263c00954bf62d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.237696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c7193a191e74360839b516e36440213', 'request_id': 'req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.256517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235894f779884573b8263c00954bf62d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.258534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235894f779884573b8263c00954bf62d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.259551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c7193a191e74360839b516e36440213', 'request_id': 'req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.267591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c7193a191e74360839b516e36440213', 'request_id': 'req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.294421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235894f779884573b8263c00954bf62d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.295787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3257b1f2-848f-4ffc-9ca8-275accf9d899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235894f779884573b8263c00954bf62d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.309336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] The token cache is being invalidate because project 0c7193a191e74360839b516e36440213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:17:56.309611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.310596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45af52c8-d0f3-4f51-9e3f-812ae681a7e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:56.326441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 863/1735] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:17:56 2022] DELETE /identity/v3/projects/0c7193a191e74360839b516e36440213 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:56.334266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af94db-56c8-4bca-9c93-6bfca1c69f4e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:56.336646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 873/1736] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:55 2022] DELETE /identity/v3/users/235894f779884573b8263c00954bf62d => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:17:56.337810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af94db-56c8-4bca-9c93-6bfca1c69f4e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:56.338105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af94db-56c8-4bca-9c93-6bfca1c69f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:56.388265 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a9af94db-56c8-4bca-9c93-6bfca1c69f4e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:56.389647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a9af94db-56c8-4bca-9c93-6bfca1c69f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:56.475105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:56.522312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 864/1737] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:56.524374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e6f41-468f-48a4-9c74-f47edc15ddee None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:56.524763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e6f41-468f-48a4-9c74-f47edc15ddee None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:56.525153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e6f41-468f-48a4-9c74-f47edc15ddee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:56.575683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-244e6f41-468f-48a4-9c74-f47edc15ddee None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:56.576870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-244e6f41-468f-48a4-9c74-f47edc15ddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:56.599860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af94db-56c8-4bca-9c93-6bfca1c69f4e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:56.600282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 874/1738] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:56.610109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af94db-56c8-4bca-9c93-6bfca1c69f4e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:56.627674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:56.628396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:56.628703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:56.628978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:56.629869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:56.629965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:56.631597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:56.685451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e6f41-468f-48a4-9c74-f47edc15ddee None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:56.688837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 865/1739] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:56.702734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e6f41-468f-48a4-9c74-f47edc15ddee None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:56.719457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd077a2d-ab15-4d1f-8900-70f3dd30533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:56.722894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd077a2d-ab15-4d1f-8900-70f3dd30533f admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:56.723215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd077a2d-ab15-4d1f-8900-70f3dd30533f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:56.726098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd077a2d-ab15-4d1f-8900-70f3dd30533f admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:56.729723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd077a2d-ab15-4d1f-8900-70f3dd30533f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:56.736538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd077a2d-ab15-4d1f-8900-70f3dd30533f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:56.832984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 875/1740] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:56 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:17:56.834921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:56.855185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b41c2-7369-42e9-89a2-47ee1321651c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:57.021406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:57.022848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.023371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.023855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.025013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:57.027231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:57.112108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 866/1741] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:56 2022] POST /identity/v3/users => generated 287 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:57.117546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd077a2d-ab15-4d1f-8900-70f3dd30533f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:57.140282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd077a2d-ab15-4d1f-8900-70f3dd30533f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:57.157872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:57.160090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:57.162753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048d483-7528-4607-8369-47d15b16124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:57.168149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048d483-7528-4607-8369-47d15b16124b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.173758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048d483-7528-4607-8369-47d15b16124b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.174368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048d483-7528-4607-8369-47d15b16124b admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.180107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048d483-7528-4607-8369-47d15b16124b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:57.185444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048d483-7528-4607-8369-47d15b16124b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:57.193898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaadc61-3930-4559-91f8-0b85cda8e45f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:57.203140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:57.203847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.204133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.204405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] PATH_INFO: `/v3/projects/284ebd2544ae4c6d8e4ff13a4330c3f8` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.205789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] RBAC: Authorizing `identity:delete_project(project_id=284ebd2544ae4c6d8e4ff13a4330c3f8)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:57.209799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:57.227570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 876/1742] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:57.229235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048d483-7528-4607-8369-47d15b16124b admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:57.257993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048d483-7528-4607-8369-47d15b16124b admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:57.283613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d9487-d20e-40fd-a8c8-4f8397b6b398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:57.284939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d9487-d20e-40fd-a8c8-4f8397b6b398 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.285441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d9487-d20e-40fd-a8c8-4f8397b6b398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.285865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d9487-d20e-40fd-a8c8-4f8397b6b398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.288139 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d9487-d20e-40fd-a8c8-4f8397b6b398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:57.298574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d9487-d20e-40fd-a8c8-4f8397b6b398 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:57.321633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 877/1743] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:57.323462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d9487-d20e-40fd-a8c8-4f8397b6b398 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:57.351990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d9487-d20e-40fd-a8c8-4f8397b6b398 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:57.389673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '284ebd2544ae4c6d8e4ff13a4330c3f8', 'request_id': 'req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:57.403170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:INSERT=3,keystone:DELETE=11,keystone:SELECT=86 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:57.417587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '284ebd2544ae4c6d8e4ff13a4330c3f8', 'request_id': 'req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:57.444839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '284ebd2544ae4c6d8e4ff13a4330c3f8', 'request_id': 'req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:57.487147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] The token cache is being invalidate because project 284ebd2544ae4c6d8e4ff13a4330c3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:17:57.488296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:57.491985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4aa54919-e7e4-4721-bab6-d6ce4dfbd46b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:57.499047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 867/1744] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:17:57 2022] DELETE /identity/v3/projects/284ebd2544ae4c6d8e4ff13a4330c3f8 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:57.501321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f5d0-2d86-41ba-8c58-262cd0006765 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.501611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f5d0-2d86-41ba-8c58-262cd0006765 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.501892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917f5d0-2d86-41ba-8c58-262cd0006765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.541166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa790d2-52d3-48b1-9c81-2ebf8091feda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:57.541853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa790d2-52d3-48b1-9c81-2ebf8091feda admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.542137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa790d2-52d3-48b1-9c81-2ebf8091feda admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.542417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa790d2-52d3-48b1-9c81-2ebf8091feda admin admin] PATH_INFO: `/v3/projects/2fd5594633b7440bb8e0da744194d323/users/3f691a508e934f92b748cc96022ac195/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.566965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0917f5d0-2d86-41ba-8c58-262cd0006765 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:57.584059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0917f5d0-2d86-41ba-8c58-262cd0006765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:57.584286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa790d2-52d3-48b1-9c81-2ebf8091feda admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd5594633b7440bb8e0da744194d323, user_id=3f691a508e934f92b748cc96022ac195, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:57.593182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa790d2-52d3-48b1-9c81-2ebf8091feda admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:57.593383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 868/1745] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:57.601955 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2f45-e9d3-48ee-924b-2aa3592a79a7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.602237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2f45-e9d3-48ee-924b-2aa3592a79a7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.602566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e2f45-e9d3-48ee-924b-2aa3592a79a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.618149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 878/1746] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:57 2022] PUT /identity/v3/projects/2fd5594633b7440bb8e0da744194d323/users/3f691a508e934f92b748cc96022ac195/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:57.620243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bf47e-ff02-4dcc-8eaa-cecc802097ef None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.620542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bf47e-ff02-4dcc-8eaa-cecc802097ef None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.623115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bf47e-ff02-4dcc-8eaa-cecc802097ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.647143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3d5e2f45-e9d3-48ee-924b-2aa3592a79a7 None None] MFA Rules not processed for user `fd0f7f51482040d7a89df3f7502c2638`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:57.648400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3d5e2f45-e9d3-48ee-924b-2aa3592a79a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:57.687789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-da6bf47e-ff02-4dcc-8eaa-cecc802097ef None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:57.689411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-da6bf47e-ff02-4dcc-8eaa-cecc802097ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:57.791470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:57.869198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 869/1747] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:57.874491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677c2b7-94de-45b2-9969-b5b8fa1009b3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.875531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677c2b7-94de-45b2-9969-b5b8fa1009b3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.876064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677c2b7-94de-45b2-9969-b5b8fa1009b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.909798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 879/1748] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:57.912093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bf47e-ff02-4dcc-8eaa-cecc802097ef None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:57.921579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-da6bf47e-ff02-4dcc-8eaa-cecc802097ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:57.947175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bf47e-ff02-4dcc-8eaa-cecc802097ef None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:57.953877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:57.954629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:57.954916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:57.955194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] PATH_INFO: `/v3/projects/feabe9d5812e46c2ae8204821517493d` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:57.955947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4677c2b7-94de-45b2-9969-b5b8fa1009b3 None None] MFA Rules not processed for user `3f691a508e934f92b748cc96022ac195`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:57.956748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] RBAC: Authorizing `identity:delete_project(project_id=feabe9d5812e46c2ae8204821517493d)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:57.959719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:57.960029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4677c2b7-94de-45b2-9969-b5b8fa1009b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:58.043529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feabe9d5812e46c2ae8204821517493d', 'request_id': 'req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.059030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feabe9d5812e46c2ae8204821517493d', 'request_id': 'req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.071924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feabe9d5812e46c2ae8204821517493d', 'request_id': 'req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.097399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] The token cache is being invalidate because project feabe9d5812e46c2ae8204821517493d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:17:58.097809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.099618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.104155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 880/1749] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:17:57 2022] DELETE /identity/v3/projects/feabe9d5812e46c2ae8204821517493d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:58.105573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:58.110732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:58.172342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 870/1750] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:17:58.178091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677c2b7-94de-45b2-9969-b5b8fa1009b3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:58.185507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4677c2b7-94de-45b2-9969-b5b8fa1009b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:58.209905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677c2b7-94de-45b2-9969-b5b8fa1009b3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:58.212949 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:58.279558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ebe83-4931-4e03-8ebb-d9f360a3485e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:58.285260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:58.287658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:58.287759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:58.289222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:58.299624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:17:58.304946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:58.309796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:58.327446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 881/1751] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:17:58.328408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:58.330569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:58.360145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a267493-8f37-4488-87f2-f49d040ac98c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:58.365411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:58.365687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe6c8-42a0-4f72-a7b4-ad69a2135ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:58.366077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:58.366334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe6c8-42a0-4f72-a7b4-ad69a2135ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:58.366612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe6c8-42a0-4f72-a7b4-ad69a2135ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:58.366879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afe6c8-42a0-4f72-a7b4-ad69a2135ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:58.367455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:58.367542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afe6c8-42a0-4f72-a7b4-ad69a2135ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:58.367705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] PATH_INFO: `/v3/users/7331e45a2e4c493f91c3b3e653d67696` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:58.369308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7331e45a2e4c493f91c3b3e653d67696)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:58.369643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afe6c8-42a0-4f72-a7b4-ad69a2135ed7 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:58.373493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:58.423322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 882/1752] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:58 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:58.424423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe6c8-42a0-4f72-a7b4-ad69a2135ed7 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:58.484218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7331e45a2e4c493f91c3b3e653d67696'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.515134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7331e45a2e4c493f91c3b3e653d67696'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.516778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7331e45a2e4c493f91c3b3e653d67696'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.519473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7331e45a2e4c493f91c3b3e653d67696'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.546213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7331e45a2e4c493f91c3b3e653d67696'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:17:58.555443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:58.598762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 871/1753] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:17:58 2022] DELETE /identity/v3/users/7331e45a2e4c493f91c3b3e653d67696 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:58.601045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:58.605579 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:58.629257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afe6c8-42a0-4f72-a7b4-ad69a2135ed7 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:58.764609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b1f42-d15b-4405-941e-2dfea4dbe854 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:58.775036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106a193-ae44-40c4-8b00-67b7019c4247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:58.775798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106a193-ae44-40c4-8b00-67b7019c4247 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:58.777002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106a193-ae44-40c4-8b00-67b7019c4247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:58.777274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106a193-ae44-40c4-8b00-67b7019c4247 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:58.778359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106a193-ae44-40c4-8b00-67b7019c4247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:58.800428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106a193-ae44-40c4-8b00-67b7019c4247 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:58.832049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:58.832825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:58.844972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:58.845552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:58.846637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:58.854557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:58.874242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 872/1754] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:58 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:58.881589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106a193-ae44-40c4-8b00-67b7019c4247 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:58.911393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106a193-ae44-40c4-8b00-67b7019c4247 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:58.979621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf864-5341-4ca5-9826-07a140cfeff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:58.981370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf864-5341-4ca5-9826-07a140cfeff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:58.981664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf864-5341-4ca5-9826-07a140cfeff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:58.982033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf864-5341-4ca5-9826-07a140cfeff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:58.982707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaf864-5341-4ca5-9826-07a140cfeff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:59.011592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaf864-5341-4ca5-9826-07a140cfeff1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:59.102874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 873/1755] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:17:59.104729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf864-5341-4ca5-9826-07a140cfeff1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:59.113736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 883/1756] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:17:58 2022] POST /identity/v3/projects => generated 367 bytes in 687 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:17:59.115613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:59.117824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:59.128349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf864-5341-4ca5-9826-07a140cfeff1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:59.175556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:59.279581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e7d15-94ee-4063-b2e6-c8b98787def1 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:59.306427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737f135-97a0-4373-b2df-4fd7093ed626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:59.308266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f135-97a0-4373-b2df-4fd7093ed626 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:59.308724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f135-97a0-4373-b2df-4fd7093ed626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:59.309148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737f135-97a0-4373-b2df-4fd7093ed626 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:59.310116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737f135-97a0-4373-b2df-4fd7093ed626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:59.312290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737f135-97a0-4373-b2df-4fd7093ed626 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:59.340707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820031b3-bbb8-45fd-815f-f928b2dd5ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:59.341659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820031b3-bbb8-45fd-815f-f928b2dd5ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:59.342093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820031b3-bbb8-45fd-815f-f928b2dd5ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:59.342514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820031b3-bbb8-45fd-815f-f928b2dd5ea5 admin admin] PATH_INFO: `/v3/projects/883f386d473e45faac379bc2482478a9/users/44a1d4294a4a48efb1b3f97db9810056/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:59.394699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820031b3-bbb8-45fd-815f-f928b2dd5ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=883f386d473e45faac379bc2482478a9, user_id=44a1d4294a4a48efb1b3f97db9810056, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:59.404437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820031b3-bbb8-45fd-815f-f928b2dd5ea5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:59.410524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 884/1757] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:17:59 2022] POST /identity/v3/users => generated 284 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:17:59.433884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737f135-97a0-4373-b2df-4fd7093ed626 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:59.440372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 874/1758] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:59 2022] PUT /identity/v3/projects/883f386d473e45faac379bc2482478a9/users/44a1d4294a4a48efb1b3f97db9810056/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:17:59.442028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820031b3-bbb8-45fd-815f-f928b2dd5ea5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:59.492389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737f135-97a0-4373-b2df-4fd7093ed626 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:59.609194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:59.639409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820031b3-bbb8-45fd-815f-f928b2dd5ea5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:59.652165 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d875-e3ca-4cfb-a4fb-4e28206f73fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:59.655037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d875-e3ca-4cfb-a4fb-4e28206f73fd admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:59.655329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d875-e3ca-4cfb-a4fb-4e28206f73fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:59.655609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151d875-e3ca-4cfb-a4fb-4e28206f73fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:59.656291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151d875-e3ca-4cfb-a4fb-4e28206f73fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:59.663668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151d875-e3ca-4cfb-a4fb-4e28206f73fd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:59.704226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 875/1759] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:59.707444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d875-e3ca-4cfb-a4fb-4e28206f73fd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:59.731673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151d875-e3ca-4cfb-a4fb-4e28206f73fd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:59.750627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058e6a7-8b92-4976-a2a1-31cf095c2885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:59.756937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573bbf6-50fd-49a3-be2c-8865faf4df1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:59.757579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573bbf6-50fd-49a3-be2c-8865faf4df1e admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:59.758207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573bbf6-50fd-49a3-be2c-8865faf4df1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:59.758207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573bbf6-50fd-49a3-be2c-8865faf4df1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:59.759133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573bbf6-50fd-49a3-be2c-8865faf4df1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:59.760401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573bbf6-50fd-49a3-be2c-8865faf4df1e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:59.764980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e6a7-8b92-4976-a2a1-31cf095c2885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:59.768977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e6a7-8b92-4976-a2a1-31cf095c2885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:59.769895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e6a7-8b92-4976-a2a1-31cf095c2885 admin admin] PATH_INFO: `/v3/projects/883f386d473e45faac379bc2482478a9/users/44a1d4294a4a48efb1b3f97db9810056/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:59.780817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058e6a7-8b92-4976-a2a1-31cf095c2885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=883f386d473e45faac379bc2482478a9, user_id=44a1d4294a4a48efb1b3f97db9810056, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:59.797866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058e6a7-8b92-4976-a2a1-31cf095c2885 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:59.801211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 885/1760] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:59.803117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573bbf6-50fd-49a3-be2c-8865faf4df1e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:59.823554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573bbf6-50fd-49a3-be2c-8865faf4df1e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:59.841463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 876/1761] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:59 2022] PUT /identity/v3/projects/883f386d473e45faac379bc2482478a9/users/44a1d4294a4a48efb1b3f97db9810056/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:17:59.856340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd116c8-858c-4457-a973-a3304b43d54c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:59.856806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd116c8-858c-4457-a973-a3304b43d54c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:59.857191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd116c8-858c-4457-a973-a3304b43d54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:59.907566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d5d57-cf77-4921-8ad3-918e0bc13293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:59.908192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d5d57-cf77-4921-8ad3-918e0bc13293 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:59.908471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d5d57-cf77-4921-8ad3-918e0bc13293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:59.908779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d5d57-cf77-4921-8ad3-918e0bc13293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:59.909425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d5d57-cf77-4921-8ad3-918e0bc13293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:59.911050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d5d57-cf77-4921-8ad3-918e0bc13293 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:59.924340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 886/1762] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:17:59.929181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d5d57-cf77-4921-8ad3-918e0bc13293 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:59.940973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d5d57-cf77-4921-8ad3-918e0bc13293 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:59.946674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53109316-14bf-450c-bc0f-b93c7d49001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:59.947326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53109316-14bf-450c-bc0f-b93c7d49001f admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:59.947713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53109316-14bf-450c-bc0f-b93c7d49001f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:59.947990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53109316-14bf-450c-bc0f-b93c7d49001f admin admin] PATH_INFO: `/v3/projects/cf626afeb71c412e84ffc0049546d3fb/users/41c400d97e4f48598b825336dda3020c/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:59.967206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53109316-14bf-450c-bc0f-b93c7d49001f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf626afeb71c412e84ffc0049546d3fb, user_id=41c400d97e4f48598b825336dda3020c, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:59.970225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53109316-14bf-450c-bc0f-b93c7d49001f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:59.976498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2dd116c8-858c-4457-a973-a3304b43d54c None None] MFA Rules not processed for user `44a1d4294a4a48efb1b3f97db9810056`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:59.981858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2dd116c8-858c-4457-a973-a3304b43d54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:00.009283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 887/1763] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:17:59 2022] PUT /identity/v3/projects/cf626afeb71c412e84ffc0049546d3fb/users/41c400d97e4f48598b825336dda3020c/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:00.024358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f92bf4-530f-49a1-8610-d6bfcc295f36 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:00.025117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f92bf4-530f-49a1-8610-d6bfcc295f36 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:00.025117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f92bf4-530f-49a1-8610-d6bfcc295f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:00.090144 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:00.103580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-04f92bf4-530f-49a1-8610-d6bfcc295f36 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:00.104867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-04f92bf4-530f-49a1-8610-d6bfcc295f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:00.314827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 877/1764] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:17:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:00.323460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2982cd7-f858-45a3-8827-68d8d68f98a1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:00.329057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2982cd7-f858-45a3-8827-68d8d68f98a1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:00.329514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2982cd7-f858-45a3-8827-68d8d68f98a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:00.340217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 888/1765] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:18:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:00.341282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f92bf4-530f-49a1-8610-d6bfcc295f36 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:00.345796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-04f92bf4-530f-49a1-8610-d6bfcc295f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:00.423259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a2982cd7-f858-45a3-8827-68d8d68f98a1 None None] MFA Rules not processed for user `41c400d97e4f48598b825336dda3020c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:00.424711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a2982cd7-f858-45a3-8827-68d8d68f98a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:00.491925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f92bf4-530f-49a1-8610-d6bfcc295f36 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:00.496321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d4619-92fa-4ef4-8e3f-8cd3795630ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:00.496977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d4619-92fa-4ef4-8e3f-8cd3795630ce service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:00.497282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d4619-92fa-4ef4-8e3f-8cd3795630ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:00.497509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d4619-92fa-4ef4-8e3f-8cd3795630ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:00.521457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d4619-92fa-4ef4-8e3f-8cd3795630ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:00.523471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d4619-92fa-4ef4-8e3f-8cd3795630ce service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:00.546797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 889/1766] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:00.606370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 878/1767] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:00.612933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d4619-92fa-4ef4-8e3f-8cd3795630ce service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:00.769336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2982cd7-f858-45a3-8827-68d8d68f98a1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:00.771602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a2982cd7-f858-45a3-8827-68d8d68f98a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:00.921843 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d4619-92fa-4ef4-8e3f-8cd3795630ce service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:00.934632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7af9-f619-429f-b2c2-b38a13fd8c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:00.937076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7af9-f619-429f-b2c2-b38a13fd8c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:00.937555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7af9-f619-429f-b2c2-b38a13fd8c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:00.938085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc7af9-f619-429f-b2c2-b38a13fd8c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:00.939645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc7af9-f619-429f-b2c2-b38a13fd8c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:00.945504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc7af9-f619-429f-b2c2-b38a13fd8c6a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:00.972662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:01.020657 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 890/1768] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:00 2022] POST /identity/v3/projects => generated 369 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:01.022581 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7af9-f619-429f-b2c2-b38a13fd8c6a admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:01.068258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:01.070545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2982cd7-f858-45a3-8827-68d8d68f98a1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:01.107751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc7af9-f619-429f-b2c2-b38a13fd8c6a admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:01.113901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:01.114764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.115046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.115315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.117864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:01.246289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:01.247357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.247649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.247931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.327653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:01.331019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:01.365575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 891/1769] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:01.369946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:01.434679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:01.436422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:01.459668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 879/1770] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:18:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:18:01.461106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:01.464331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:01.549814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eecb7-e6e7-4791-b415-15b334c6b24f service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:01.563595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc8340-1cc0-4318-a60c-32ceefb8a91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:01.564706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc8340-1cc0-4318-a60c-32ceefb8a91d admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.565162 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc8340-1cc0-4318-a60c-32ceefb8a91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.565578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc8340-1cc0-4318-a60c-32ceefb8a91d admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.566390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc8340-1cc0-4318-a60c-32ceefb8a91d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:01.569376 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc8340-1cc0-4318-a60c-32ceefb8a91d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:01.668997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 892/1771] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:01 2022] POST /identity/v3/users => generated 291 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:01.675034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc8340-1cc0-4318-a60c-32ceefb8a91d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:01.715005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc8340-1cc0-4318-a60c-32ceefb8a91d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:01.734516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad513f-88c6-4fb5-a80a-065d12d8927d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:01.735165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad513f-88c6-4fb5-a80a-065d12d8927d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.735435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad513f-88c6-4fb5-a80a-065d12d8927d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.735707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad513f-88c6-4fb5-a80a-065d12d8927d admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.742132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad513f-88c6-4fb5-a80a-065d12d8927d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:01.757729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad513f-88c6-4fb5-a80a-065d12d8927d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:01.790318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dc618-21e4-4d86-922a-24121051987a service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:01.796601 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 893/1772] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:01.801627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad513f-88c6-4fb5-a80a-065d12d8927d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:01.812169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:01.813164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.814854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.815864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] PATH_INFO: `/v3/projects/d498115951cd4bd5bb4bee9da2d8d123` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.822566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d498115951cd4bd5bb4bee9da2d8d123)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:01.832665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:01.838419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad513f-88c6-4fb5-a80a-065d12d8927d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:01.860427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df875527-ad2c-4d43-9473-44d6dc13a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:01.861216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df875527-ad2c-4d43-9473-44d6dc13a395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:01.861564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df875527-ad2c-4d43-9473-44d6dc13a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:01.863279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df875527-ad2c-4d43-9473-44d6dc13a395 admin admin] PATH_INFO: `/v3/projects/fb93db714aee409b8d062a2297fa3ea9/users/bb967e1f8c434ff2bb7befe1aa107e6e/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:01.902786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df875527-ad2c-4d43-9473-44d6dc13a395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb93db714aee409b8d062a2297fa3ea9, user_id=bb967e1f8c434ff2bb7befe1aa107e6e, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:01.909395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df875527-ad2c-4d43-9473-44d6dc13a395 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:01.944240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 894/1773] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:01 2022] PUT /identity/v3/projects/fb93db714aee409b8d062a2297fa3ea9/users/bb967e1f8c434ff2bb7befe1aa107e6e/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:01.947813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df875527-ad2c-4d43-9473-44d6dc13a395 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:01.970677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:02.056814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:02.147172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd498115951cd4bd5bb4bee9da2d8d123', 'request_id': 'req-af79b0f4-b363-432a-80d0-2a1322339b79'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:02.149025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df875527-ad2c-4d43-9473-44d6dc13a395 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:02.188925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd498115951cd4bd5bb4bee9da2d8d123', 'request_id': 'req-af79b0f4-b363-432a-80d0-2a1322339b79'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:02.206321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd498115951cd4bd5bb4bee9da2d8d123', 'request_id': 'req-af79b0f4-b363-432a-80d0-2a1322339b79'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:02.233392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] The token cache is being invalidate because project d498115951cd4bd5bb4bee9da2d8d123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:02.233625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:02.237057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af79b0f4-b363-432a-80d0-2a1322339b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:02.241675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 880/1774] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:01 2022] DELETE /identity/v3/projects/d498115951cd4bd5bb4bee9da2d8d123 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:02.247717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5483-277d-460d-8dad-0e2ac9c03024 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:02.248016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5483-277d-460d-8dad-0e2ac9c03024 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:02.248455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5483-277d-460d-8dad-0e2ac9c03024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:02.305200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a98f5483-277d-460d-8dad-0e2ac9c03024 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:02.307353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a98f5483-277d-460d-8dad-0e2ac9c03024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:02.341688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:02.342318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:02.345021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:02.345333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:02.346041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:02.347731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:02.386632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 895/1775] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:18:02.388350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:02.390399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:02.413131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c98f34-01c8-4a99-917e-2a974cb50efc admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:02.443000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d5ae1-c1c4-4986-a8e2-6f2c30be15d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:02.445365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d5ae1-c1c4-4986-a8e2-6f2c30be15d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:02.448789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d5ae1-c1c4-4986-a8e2-6f2c30be15d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:02.449211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d5ae1-c1c4-4986-a8e2-6f2c30be15d4 admin admin] PATH_INFO: `/v3/projects/fb93db714aee409b8d062a2297fa3ea9/users/bb967e1f8c434ff2bb7befe1aa107e6e/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:02.451810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d5ae1-c1c4-4986-a8e2-6f2c30be15d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb93db714aee409b8d062a2297fa3ea9, user_id=bb967e1f8c434ff2bb7befe1aa107e6e, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:02.454754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d5ae1-c1c4-4986-a8e2-6f2c30be15d4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:02.482663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 896/1776] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:02 2022] PUT /identity/v3/projects/fb93db714aee409b8d062a2297fa3ea9/users/bb967e1f8c434ff2bb7befe1aa107e6e/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:02.494582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490009e-c9b6-43a5-a140-7d0330029464 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:02.495110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490009e-c9b6-43a5-a140-7d0330029464 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:02.495542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490009e-c9b6-43a5-a140-7d0330029464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:02.511134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 881/1777] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:02.521921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014ef29-2887-42c9-ba08-182b340451dd None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:02.522205 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014ef29-2887-42c9-ba08-182b340451dd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:02.524516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014ef29-2887-42c9-ba08-182b340451dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:02.545056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8490009e-c9b6-43a5-a140-7d0330029464 None None] MFA Rules not processed for user `bb967e1f8c434ff2bb7befe1aa107e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:02.546369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8490009e-c9b6-43a5-a140-7d0330029464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:02.580533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3014ef29-2887-42c9-ba08-182b340451dd None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:02.585802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3014ef29-2887-42c9-ba08-182b340451dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:02.757431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 882/1778] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:02.767019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 897/1779] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:02.776800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014ef29-2887-42c9-ba08-182b340451dd None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:02.789850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490009e-c9b6-43a5-a140-7d0330029464 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:02.791682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8490009e-c9b6-43a5-a140-7d0330029464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:02.794266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014ef29-2887-42c9-ba08-182b340451dd None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:02.808367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:02.809219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:02.809509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:02.809839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:02.810541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:02.812781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:02.814539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:02.846628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 883/1780] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:02.851423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:02.871498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aec89-61a3-481d-b743-d4ec54bdbef3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:02.885284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d5642-a018-4695-9f41-e207275d8344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:02.886985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d5642-a018-4695-9f41-e207275d8344 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:02.887273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d5642-a018-4695-9f41-e207275d8344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:02.887549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d5642-a018-4695-9f41-e207275d8344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:02.890790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d5642-a018-4695-9f41-e207275d8344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:02.897241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d5642-a018-4695-9f41-e207275d8344 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:03.008476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 884/1781] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:02 2022] POST /identity/v3/projects => generated 407 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:03.012692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d5642-a018-4695-9f41-e207275d8344 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:03.031625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490009e-c9b6-43a5-a140-7d0330029464 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:03.044723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:03.062926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:03.188126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d5642-a018-4695-9f41-e207275d8344 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:03.197533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f40db-3c69-4e8a-b582-162ac0882223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:03.198743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f40db-3c69-4e8a-b582-162ac0882223 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.199185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f40db-3c69-4e8a-b582-162ac0882223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.199610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f40db-3c69-4e8a-b582-162ac0882223 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.200403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f40db-3c69-4e8a-b582-162ac0882223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:03.202984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f40db-3c69-4e8a-b582-162ac0882223 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:03.245271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60f8b0-e040-4196-967d-26a2b275923c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:03.246274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f8b0-e040-4196-967d-26a2b275923c service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.246719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f8b0-e040-4196-967d-26a2b275923c service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.247671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60f8b0-e040-4196-967d-26a2b275923c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.249964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 885/1782] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:03 2022] POST /identity/v3/users => generated 304 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:03.261234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f40db-3c69-4e8a-b582-162ac0882223 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:03.329207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f40db-3c69-4e8a-b582-162ac0882223 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:03.353319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469295c3-6abb-4976-a46a-f6080574bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:03.355116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469295c3-6abb-4976-a46a-f6080574bb72 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.356842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469295c3-6abb-4976-a46a-f6080574bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.358692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469295c3-6abb-4976-a46a-f6080574bb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.362772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469295c3-6abb-4976-a46a-f6080574bb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:03.393846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469295c3-6abb-4976-a46a-f6080574bb72 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:03.451314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 886/1783] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:03.453823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469295c3-6abb-4976-a46a-f6080574bb72 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:03.472146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60f8b0-e040-4196-967d-26a2b275923c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:03.474299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60f8b0-e040-4196-967d-26a2b275923c service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:03.495832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469295c3-6abb-4976-a46a-f6080574bb72 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:03.513322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b6cf4-1b6d-4c5d-bf01-3b2ff18ccfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:03.513322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b6cf4-1b6d-4c5d-bf01-3b2ff18ccfcd admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.513618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b6cf4-1b6d-4c5d-bf01-3b2ff18ccfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.513705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b6cf4-1b6d-4c5d-bf01-3b2ff18ccfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.514033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 898/1784] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:03.523399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b6cf4-1b6d-4c5d-bf01-3b2ff18ccfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:03.527826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b6cf4-1b6d-4c5d-bf01-3b2ff18ccfcd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:03.580823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 887/1785] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:03.585469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60f8b0-e040-4196-967d-26a2b275923c service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:03.607895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60f8b0-e040-4196-967d-26a2b275923c service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:03.632013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbf2a-46e6-4ca6-9414-ae38a9a4b7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:03.632937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbf2a-46e6-4ca6-9414-ae38a9a4b7ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.633532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbf2a-46e6-4ca6-9414-ae38a9a4b7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.633908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbf2a-46e6-4ca6-9414-ae38a9a4b7ee admin admin] PATH_INFO: `/v3/projects/201840de51b645e7afa0516404e61232/users/e21010cb09c049bf87d178f95ed5f44f/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.675741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fbf2a-46e6-4ca6-9414-ae38a9a4b7ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=201840de51b645e7afa0516404e61232, user_id=e21010cb09c049bf87d178f95ed5f44f, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:03.678974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fbf2a-46e6-4ca6-9414-ae38a9a4b7ee admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:03.719297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 899/1786] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:03 2022] PUT /identity/v3/projects/201840de51b645e7afa0516404e61232/users/e21010cb09c049bf87d178f95ed5f44f/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:03.728072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655214a-8c07-4e1e-a155-60359f12dfa9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:03.728609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655214a-8c07-4e1e-a155-60359f12dfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:03.729076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655214a-8c07-4e1e-a155-60359f12dfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:03.838674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6655214a-8c07-4e1e-a155-60359f12dfa9 None None] MFA Rules not processed for user `e21010cb09c049bf87d178f95ed5f44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:03.845066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6655214a-8c07-4e1e-a155-60359f12dfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:04.130840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 888/1787] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:03 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:04.135463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbf2a-46e6-4ca6-9414-ae38a9a4b7ee admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:04.475863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbf2a-46e6-4ca6-9414-ae38a9a4b7ee admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:04.494366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6e5c-e781-48c7-873e-eb68dbb0c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:04.498053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6e5c-e781-48c7-873e-eb68dbb0c4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:04.499800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6e5c-e781-48c7-873e-eb68dbb0c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:04.501617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6e5c-e781-48c7-873e-eb68dbb0c4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:04.505520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c6e5c-e781-48c7-873e-eb68dbb0c4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:04.511868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c6e5c-e781-48c7-873e-eb68dbb0c4c8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:04.585660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748ed1a-aaaf-416a-99a7-73eaff4c19c0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:04.590129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748ed1a-aaaf-416a-99a7-73eaff4c19c0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:04.590545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748ed1a-aaaf-416a-99a7-73eaff4c19c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:04.633158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 900/1788] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:04 2022] POST /identity/v3/projects => generated 407 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:04.633939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d748ed1a-aaaf-416a-99a7-73eaff4c19c0 None None] MFA Rules not processed for user `5f1d5c29df30465ead9834b63c3e93bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:04.635355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d748ed1a-aaaf-416a-99a7-73eaff4c19c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:04.641781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6e5c-e781-48c7-873e-eb68dbb0c4c8 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:04.706801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:04.779207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 889/1789] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Aug 31 19:18:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:04.921546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6e5c-e781-48c7-873e-eb68dbb0c4c8 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:04.962095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:04.964199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:04.964508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:04.965071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:04.969213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:04.972528 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:05.083446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 901/1790] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:04 2022] POST /identity/v3/users => generated 310 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:18:05.084924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748ed1a-aaaf-416a-99a7-73eaff4c19c0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:05.109582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748ed1a-aaaf-416a-99a7-73eaff4c19c0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:05.148360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:05.152316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:05.171933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e0a8d-7b85-4559-b227-99fff3c0b3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:05.171933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e0a8d-7b85-4559-b227-99fff3c0b3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:05.171933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e0a8d-7b85-4559-b227-99fff3c0b3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:05.171933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e0a8d-7b85-4559-b227-99fff3c0b3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:05.172238 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e0a8d-7b85-4559-b227-99fff3c0b3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:05.173569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e0a8d-7b85-4559-b227-99fff3c0b3c7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:05.194658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62856d4-9b49-4ccb-8650-a857e2c5efed admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:05.214396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3c555-3cf4-4c3f-9308-c7dac9e528cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:05.214898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c555-3cf4-4c3f-9308-c7dac9e528cd service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:05.215347 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c555-3cf4-4c3f-9308-c7dac9e528cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:05.215584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c555-3cf4-4c3f-9308-c7dac9e528cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:05.231634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 890/1791] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:05.237457 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e0a8d-7b85-4559-b227-99fff3c0b3c7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:05.262524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3c555-3cf4-4c3f-9308-c7dac9e528cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:05.263994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3c555-3cf4-4c3f-9308-c7dac9e528cd service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:05.284050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e0a8d-7b85-4559-b227-99fff3c0b3c7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:05.287391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 902/1792] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:05.305458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b06ef-8562-49d3-932c-16445beaf1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:05.306548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b06ef-8562-49d3-932c-16445beaf1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:05.307054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b06ef-8562-49d3-932c-16445beaf1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:05.307489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b06ef-8562-49d3-932c-16445beaf1ec admin admin] PATH_INFO: `/v3/projects/07aab95d66a04bd3ad1cf29ebbc6624a/users/5dd4f5e098594e7eb99e4a8c8551847a/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:05.314300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:05.346296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b06ef-8562-49d3-932c-16445beaf1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=07aab95d66a04bd3ad1cf29ebbc6624a, user_id=5dd4f5e098594e7eb99e4a8c8551847a, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:05.349796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b06ef-8562-49d3-932c-16445beaf1ec admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:05.398464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 891/1793] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:05 2022] PUT /identity/v3/projects/07aab95d66a04bd3ad1cf29ebbc6624a/users/5dd4f5e098594e7eb99e4a8c8551847a/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:05.402205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3c555-3cf4-4c3f-9308-c7dac9e528cd service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:05.453925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3c555-3cf4-4c3f-9308-c7dac9e528cd service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:05.720301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c149e-4d3d-4791-bab0-9294028c946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:05.722105 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c149e-4d3d-4791-bab0-9294028c946d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:05.722804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c149e-4d3d-4791-bab0-9294028c946d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:05.723349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c149e-4d3d-4791-bab0-9294028c946d admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:05.724395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c149e-4d3d-4791-bab0-9294028c946d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:05.729993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c149e-4d3d-4791-bab0-9294028c946d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:05.780765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 903/1794] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:05.781426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b06ef-8562-49d3-932c-16445beaf1ec admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:05.801512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b06ef-8562-49d3-932c-16445beaf1ec admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:05.827507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:05.830562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:05.830973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:05.831250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] PATH_INFO: `/v3/projects/07aab95d66a04bd3ad1cf29ebbc6624a/users/5dd4f5e098594e7eb99e4a8c8551847a/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:05.843504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=07aab95d66a04bd3ad1cf29ebbc6624a, user_id=5dd4f5e098594e7eb99e4a8c8551847a, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:05.865502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:05.910502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 892/1795] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:05 2022] PUT /identity/v3/projects/07aab95d66a04bd3ad1cf29ebbc6624a/users/5dd4f5e098594e7eb99e4a8c8551847a/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:05.922356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe468a60-484f-42d4-9b4d-406e273ebeb1 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:05.922843 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe468a60-484f-42d4-9b4d-406e273ebeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:05.923274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe468a60-484f-42d4-9b4d-406e273ebeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:06.010488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fe468a60-484f-42d4-9b4d-406e273ebeb1 None None] MFA Rules not processed for user `5dd4f5e098594e7eb99e4a8c8551847a`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:06.012485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fe468a60-484f-42d4-9b4d-406e273ebeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:06.169364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 904/1796] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:05 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:06.221078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:06.223024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:06.482867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2017b-6d9b-49bc-93dc-b287bf7b5ddd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:06.512381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384da72-652c-4b23-b883-3d6d79f8d4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:06.516705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384da72-652c-4b23-b883-3d6d79f8d4fe service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:06.517083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384da72-652c-4b23-b883-3d6d79f8d4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:06.518395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384da72-652c-4b23-b883-3d6d79f8d4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:06.568692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384da72-652c-4b23-b883-3d6d79f8d4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:06.573282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384da72-652c-4b23-b883-3d6d79f8d4fe service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:06.618238 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 893/1797] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:06 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:06.955486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe468a60-484f-42d4-9b4d-406e273ebeb1 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:06.972695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe468a60-484f-42d4-9b4d-406e273ebeb1 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:06.988319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed364f-c130-44f9-9e6d-e98a6b478f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:06.990120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed364f-c130-44f9-9e6d-e98a6b478f3c service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:06.990950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed364f-c130-44f9-9e6d-e98a6b478f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:06.992003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed364f-c130-44f9-9e6d-e98a6b478f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:07.045123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:07.198000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed364f-c130-44f9-9e6d-e98a6b478f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:07.199677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed364f-c130-44f9-9e6d-e98a6b478f3c service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:07.229436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 905/1798] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:06 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:07.890555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384da72-652c-4b23-b883-3d6d79f8d4fe service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:08.046528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384da72-652c-4b23-b883-3d6d79f8d4fe service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:08.083904 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:08.085365 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:08.086520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:08.087013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] PATH_INFO: `/v3/users/41c400d97e4f48598b825336dda3020c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:08.089111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c400d97e4f48598b825336dda3020c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:08.091478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:08.209194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c400d97e4f48598b825336dda3020c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:08.226348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7eeb-b474-4ea2-929e-209a6cb8526b None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:08.226897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7eeb-b474-4ea2-929e-209a6cb8526b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:08.227914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7eeb-b474-4ea2-929e-209a6cb8526b None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:08.229872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 906/1799] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:08.247106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae3e01-9afd-4b06-8ac9-32ce9417a274 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:08.247569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae3e01-9afd-4b06-8ac9-32ce9417a274 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:08.247956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae3e01-9afd-4b06-8ac9-32ce9417a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:08.269818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c400d97e4f48598b825336dda3020c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:08.272093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c400d97e4f48598b825336dda3020c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:08.273974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c400d97e4f48598b825336dda3020c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:08.297978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-48ae3e01-9afd-4b06-8ac9-32ce9417a274 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:08.302447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-48ae3e01-9afd-4b06-8ac9-32ce9417a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:08.304622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-33941382-fa05-4c92-bc1c-d51d4a2397d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c400d97e4f48598b825336dda3020c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:08.308922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:08.344480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 894/1800] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:07 2022] DELETE /identity/v3/users/41c400d97e4f48598b825336dda3020c => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:08.348976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc4302-46ee-44ba-bfe1-b774e7a60d68 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:08.349922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc4302-46ee-44ba-bfe1-b774e7a60d68 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:08.350862 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc4302-46ee-44ba-bfe1-b774e7a60d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:08.417138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-64bc4302-46ee-44ba-bfe1-b774e7a60d68 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:08.419853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-64bc4302-46ee-44ba-bfe1-b774e7a60d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:08.459865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 907/1801] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:18:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:08.485121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae3e01-9afd-4b06-8ac9-32ce9417a274 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:08.487488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-48ae3e01-9afd-4b06-8ac9-32ce9417a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:08.504693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae3e01-9afd-4b06-8ac9-32ce9417a274 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:08.566419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 895/1802] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:08.575723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc4302-46ee-44ba-bfe1-b774e7a60d68 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:08.577613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-64bc4302-46ee-44ba-bfe1-b774e7a60d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:08.619713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:08.621618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:08.622031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:08.622303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:08.640556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:08.642275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:08.655512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 908/1803] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:08.711316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc4302-46ee-44ba-bfe1-b774e7a60d68 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:08.715780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15778ccd-9819-42fb-8b98-4b7e0f8ba6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:08.716737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778ccd-9819-42fb-8b98-4b7e0f8ba6bc service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:08.717179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778ccd-9819-42fb-8b98-4b7e0f8ba6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:08.717458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15778ccd-9819-42fb-8b98-4b7e0f8ba6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:08.740846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15778ccd-9819-42fb-8b98-4b7e0f8ba6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:08.744778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15778ccd-9819-42fb-8b98-4b7e0f8ba6bc service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:08.760124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 896/1804] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:09.129493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:09.129493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:09.153232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ee5b2-0980-4407-8665-e0885c51cb2f service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:09.188798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:09.189454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:09.189734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:09.190256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] PATH_INFO: `/v3/projects/cf626afeb71c412e84ffc0049546d3fb` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:09.191432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf626afeb71c412e84ffc0049546d3fb)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:09.194002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:09.253240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:09.276645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf626afeb71c412e84ffc0049546d3fb', 'request_id': 'req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.283874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf626afeb71c412e84ffc0049546d3fb', 'request_id': 'req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.293814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf626afeb71c412e84ffc0049546d3fb', 'request_id': 'req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.314999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] The token cache is being invalidate because project cf626afeb71c412e84ffc0049546d3fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:09.315389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.316284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.318881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 909/1805] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:09 2022] DELETE /identity/v3/projects/cf626afeb71c412e84ffc0049546d3fb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:09.322904 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15778ccd-9819-42fb-8b98-4b7e0f8ba6bc service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:09.334087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15778ccd-9819-42fb-8b98-4b7e0f8ba6bc service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:09.418346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:09.419169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:09.419580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:09.419971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] PATH_INFO: `/v3/users/bb967e1f8c434ff2bb7befe1aa107e6e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:09.425316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb967e1f8c434ff2bb7befe1aa107e6e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:09.428893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:09.488702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:09.504811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb967e1f8c434ff2bb7befe1aa107e6e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.525342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb967e1f8c434ff2bb7befe1aa107e6e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.525544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb967e1f8c434ff2bb7befe1aa107e6e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.529971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb967e1f8c434ff2bb7befe1aa107e6e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.539961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-daadbef2-c6f4-431c-95c8-5e1c48033930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb967e1f8c434ff2bb7befe1aa107e6e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:09.560738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 897/1806] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:09 2022] DELETE /identity/v3/users/bb967e1f8c434ff2bb7befe1aa107e6e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:10.026406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:10.028319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:10.045922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd69bd-67ee-4fd9-9706-ecb7b7b05746 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:10.108455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:10.109126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.109402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.109673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] PATH_INFO: `/v3/projects/fb93db714aee409b8d062a2297fa3ea9` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.110819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb93db714aee409b8d062a2297fa3ea9)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:10.112763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:10.167709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb93db714aee409b8d062a2297fa3ea9', 'request_id': 'req-f002767a-6b90-46b2-9d58-07f642c35186'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:10.175990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb93db714aee409b8d062a2297fa3ea9', 'request_id': 'req-f002767a-6b90-46b2-9d58-07f642c35186'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:10.183791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb93db714aee409b8d062a2297fa3ea9', 'request_id': 'req-f002767a-6b90-46b2-9d58-07f642c35186'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:10.206079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] The token cache is being invalidate because project fb93db714aee409b8d062a2297fa3ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:10.206463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:10.207809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f002767a-6b90-46b2-9d58-07f642c35186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:10.227253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 910/1807] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:10 2022] DELETE /identity/v3/projects/fb93db714aee409b8d062a2297fa3ea9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:10.228076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47bfb-dadf-4058-88ef-d2e3dd9c4228 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.228366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47bfb-dadf-4058-88ef-d2e3dd9c4228 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.229083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47bfb-dadf-4058-88ef-d2e3dd9c4228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.253814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e3d47bfb-dadf-4058-88ef-d2e3dd9c4228 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:10.255301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e3d47bfb-dadf-4058-88ef-d2e3dd9c4228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:10.412619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 898/1808] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:10.413512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeb250-69f6-43f4-8511-a7fc07485b99 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.413512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeb250-69f6-43f4-8511-a7fc07485b99 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.413512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeb250-69f6-43f4-8511-a7fc07485b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.440786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-06aeb250-69f6-43f4-8511-a7fc07485b99 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:10.441906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-06aeb250-69f6-43f4-8511-a7fc07485b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:10.460442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 911/1809] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:10.461541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d47bfb-dadf-4058-88ef-d2e3dd9c4228 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:10.487932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d47bfb-dadf-4058-88ef-d2e3dd9c4228 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:10.496847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:10.500154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.500535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.500840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.501709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:10.502019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:10.510722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:10.553216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeb250-69f6-43f4-8511-a7fc07485b99 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:10.569051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 899/1810] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:10.573207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeb250-69f6-43f4-8511-a7fc07485b99 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:10.604926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495284f-687e-48fa-b621-5355d2e33779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:10.605809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495284f-687e-48fa-b621-5355d2e33779 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.607546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495284f-687e-48fa-b621-5355d2e33779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.607997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495284f-687e-48fa-b621-5355d2e33779 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.608903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495284f-687e-48fa-b621-5355d2e33779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:10.611096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495284f-687e-48fa-b621-5355d2e33779 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:10.667448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 912/1811] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:10 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:10.671763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:10.699588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66de42-913d-41a6-ad29-9dc7d04f5e0b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:10.844878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09703b20-abb7-4fa6-bd9f-6a429c9dc8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:10.845955 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09703b20-abb7-4fa6-bd9f-6a429c9dc8fe admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.846854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09703b20-abb7-4fa6-bd9f-6a429c9dc8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.847530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09703b20-abb7-4fa6-bd9f-6a429c9dc8fe admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.848558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09703b20-abb7-4fa6-bd9f-6a429c9dc8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:10.851373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09703b20-abb7-4fa6-bd9f-6a429c9dc8fe admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:10.894567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 900/1812] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:10 2022] POST /identity/v3/users => generated 296 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:10.897983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495284f-687e-48fa-b621-5355d2e33779 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:10.918506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495284f-687e-48fa-b621-5355d2e33779 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:10.932457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e480d-8ad2-4d33-8677-00012ae5f2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:10.933123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e480d-8ad2-4d33-8677-00012ae5f2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:10.933404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e480d-8ad2-4d33-8677-00012ae5f2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:10.933680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e480d-8ad2-4d33-8677-00012ae5f2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:10.936893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e480d-8ad2-4d33-8677-00012ae5f2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:10.938934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e480d-8ad2-4d33-8677-00012ae5f2cd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:10.955415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 913/1813] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:10.959340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09703b20-abb7-4fa6-bd9f-6a429c9dc8fe admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:10.980506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09703b20-abb7-4fa6-bd9f-6a429c9dc8fe admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:11.008471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca89006-cebc-444b-b038-43bae00a745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:11.009140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89006-cebc-444b-b038-43bae00a745f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:11.009426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89006-cebc-444b-b038-43bae00a745f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:11.009701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89006-cebc-444b-b038-43bae00a745f admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:11.010361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca89006-cebc-444b-b038-43bae00a745f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:11.012150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca89006-cebc-444b-b038-43bae00a745f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:11.035359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 901/1814] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:18:11.041185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e480d-8ad2-4d33-8677-00012ae5f2cd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:11.056430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e480d-8ad2-4d33-8677-00012ae5f2cd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:11.062083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0921-4ff8-498b-9b9c-4f92b5612b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:11.063433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0921-4ff8-498b-9b9c-4f92b5612b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:11.063916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0921-4ff8-498b-9b9c-4f92b5612b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:11.064467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0921-4ff8-498b-9b9c-4f92b5612b10 admin admin] PATH_INFO: `/v3/projects/b998ed7178a7459db8ccbf9c38877497/users/546aad9b0cf5449abd8b7f1e06da8b5e/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:11.068509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:11.088717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0921-4ff8-498b-9b9c-4f92b5612b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b998ed7178a7459db8ccbf9c38877497, user_id=546aad9b0cf5449abd8b7f1e06da8b5e, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:11.093144 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0921-4ff8-498b-9b9c-4f92b5612b10 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:11.118001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 914/1815] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:11 2022] PUT /identity/v3/projects/b998ed7178a7459db8ccbf9c38877497/users/546aad9b0cf5449abd8b7f1e06da8b5e/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:11.128863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fe46c-6364-40b1-87fa-e6985831e887 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:11.129287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fe46c-6364-40b1-87fa-e6985831e887 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:11.129637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fe46c-6364-40b1-87fa-e6985831e887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:11.187312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c19fe46c-6364-40b1-87fa-e6985831e887 None None] MFA Rules not processed for user `546aad9b0cf5449abd8b7f1e06da8b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:11.189026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c19fe46c-6364-40b1-87fa-e6985831e887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:11.199876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:11.301356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 902/1816] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:11.306078 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0921-4ff8-498b-9b9c-4f92b5612b10 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:11.432922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0921-4ff8-498b-9b9c-4f92b5612b10 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:11.445299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8615e-bfe2-4ce7-9a43-c505107fc0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:11.446211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8615e-bfe2-4ce7-9a43-c505107fc0aa admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:11.446791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8615e-bfe2-4ce7-9a43-c505107fc0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:11.447353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8615e-bfe2-4ce7-9a43-c505107fc0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:11.448280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8615e-bfe2-4ce7-9a43-c505107fc0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:11.451102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8615e-bfe2-4ce7-9a43-c505107fc0aa admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:11.525463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 915/1817] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:11 2022] POST /identity/v3/projects => generated 391 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:11.529215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fe46c-6364-40b1-87fa-e6985831e887 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:11.554820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fe46c-6364-40b1-87fa-e6985831e887 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:11.705528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9cb9a-9b73-48f3-abe9-5345ce983741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:11.707454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9cb9a-9b73-48f3-abe9-5345ce983741 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:11.709016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9cb9a-9b73-48f3-abe9-5345ce983741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:11.709332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9cb9a-9b73-48f3-abe9-5345ce983741 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:11.710064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9cb9a-9b73-48f3-abe9-5345ce983741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:11.713494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9cb9a-9b73-48f3-abe9-5345ce983741 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:11.753703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8615e-bfe2-4ce7-9a43-c505107fc0aa admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:11.755074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 903/1818] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:11 2022] POST /identity/v3/users => generated 302 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:11.776226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8615e-bfe2-4ce7-9a43-c505107fc0aa admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:11.791617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319b62d-fa9b-4f04-803f-b4a180a56b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:11.792802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319b62d-fa9b-4f04-803f-b4a180a56b93 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:11.793976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319b62d-fa9b-4f04-803f-b4a180a56b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:11.794638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319b62d-fa9b-4f04-803f-b4a180a56b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:11.795520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319b62d-fa9b-4f04-803f-b4a180a56b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:11.797652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319b62d-fa9b-4f04-803f-b4a180a56b93 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:11.825526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 916/1819] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:18:11.825701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9cb9a-9b73-48f3-abe9-5345ce983741 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:11.861624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9cb9a-9b73-48f3-abe9-5345ce983741 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:11.880544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba880b-73ba-4072-8184-a1ed21c8e536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:11.883561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba880b-73ba-4072-8184-a1ed21c8e536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:11.884101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba880b-73ba-4072-8184-a1ed21c8e536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:11.894609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba880b-73ba-4072-8184-a1ed21c8e536 admin admin] PATH_INFO: `/v3/projects/87fb4e50ed144393badee4cacc838f54/users/bbd464bc69354cd08db08049e69750ee/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:11.923310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba880b-73ba-4072-8184-a1ed21c8e536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fb4e50ed144393badee4cacc838f54, user_id=bbd464bc69354cd08db08049e69750ee, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:11.926410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba880b-73ba-4072-8184-a1ed21c8e536 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:11.948079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 904/1820] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:11 2022] PUT /identity/v3/projects/87fb4e50ed144393badee4cacc838f54/users/bbd464bc69354cd08db08049e69750ee/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:11.952864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319b62d-fa9b-4f04-803f-b4a180a56b93 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:11.971851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319b62d-fa9b-4f04-803f-b4a180a56b93 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:12.065436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37661aaf-a68e-4f82-a254-9b2885b5e965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:12.066141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37661aaf-a68e-4f82-a254-9b2885b5e965 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:12.066429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37661aaf-a68e-4f82-a254-9b2885b5e965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:12.066706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37661aaf-a68e-4f82-a254-9b2885b5e965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:12.068062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37661aaf-a68e-4f82-a254-9b2885b5e965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:12.069863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37661aaf-a68e-4f82-a254-9b2885b5e965 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:12.080196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 917/1821] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:12.084829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba880b-73ba-4072-8184-a1ed21c8e536 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:12.095197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba880b-73ba-4072-8184-a1ed21c8e536 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:12.100924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:12.101586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:12.101867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:12.102147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] PATH_INFO: `/v3/projects/87fb4e50ed144393badee4cacc838f54/users/bbd464bc69354cd08db08049e69750ee/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:12.104273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fb4e50ed144393badee4cacc838f54, user_id=bbd464bc69354cd08db08049e69750ee, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:12.107005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:12.125911 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 905/1822] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:12 2022] PUT /identity/v3/projects/87fb4e50ed144393badee4cacc838f54/users/bbd464bc69354cd08db08049e69750ee/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:12.137872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04678b-4c1d-4d09-8109-66a050856d4d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:12.138161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04678b-4c1d-4d09-8109-66a050856d4d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:12.138433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04678b-4c1d-4d09-8109-66a050856d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:12.181236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-cb04678b-4c1d-4d09-8109-66a050856d4d None None] MFA Rules not processed for user `bbd464bc69354cd08db08049e69750ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:12.182516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cb04678b-4c1d-4d09-8109-66a050856d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:12.263809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:12.310446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 918/1823] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:12 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:12.322167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:12.324208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:12.409138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae7946-18d8-4237-97f6-5a45cd3af6bc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:12.418699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f5ff0-9d4e-4e58-8db2-09feb9ce6c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:12.419336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f5ff0-9d4e-4e58-8db2-09feb9ce6c4d service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:12.419617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f5ff0-9d4e-4e58-8db2-09feb9ce6c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:12.419892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f5ff0-9d4e-4e58-8db2-09feb9ce6c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:12.432239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f5ff0-9d4e-4e58-8db2-09feb9ce6c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:12.434124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f5ff0-9d4e-4e58-8db2-09feb9ce6c4d service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:12.446448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 906/1824] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:12 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:12.512008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04678b-4c1d-4d09-8109-66a050856d4d None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:12.513608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cb04678b-4c1d-4d09-8109-66a050856d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:12.527028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04678b-4c1d-4d09-8109-66a050856d4d None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:12.530756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f737a-b7b6-4d14-b788-5a0744f45abc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:12.531636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f737a-b7b6-4d14-b788-5a0744f45abc service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:12.531917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f737a-b7b6-4d14-b788-5a0744f45abc service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:12.532233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f737a-b7b6-4d14-b788-5a0744f45abc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:12.601903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f737a-b7b6-4d14-b788-5a0744f45abc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:12.603652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f737a-b7b6-4d14-b788-5a0744f45abc service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:12.613542 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 919/1825] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:18.423758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f5ff0-9d4e-4e58-8db2-09feb9ce6c4d service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:18.444590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:18.544989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f5ff0-9d4e-4e58-8db2-09feb9ce6c4d service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:18.550215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:18.550892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:18.555772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:18.556395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] PATH_INFO: `/v3/users/546aad9b0cf5449abd8b7f1e06da8b5e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:18.558011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546aad9b0cf5449abd8b7f1e06da8b5e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:18.560400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:18.668823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546aad9b0cf5449abd8b7f1e06da8b5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:18.690902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546aad9b0cf5449abd8b7f1e06da8b5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:18.692318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546aad9b0cf5449abd8b7f1e06da8b5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:18.693637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546aad9b0cf5449abd8b7f1e06da8b5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:18.710219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546aad9b0cf5449abd8b7f1e06da8b5e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:18.735991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e815a1d-d215-4382-94d2-54002c9c6897 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:18.736419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e815a1d-d215-4382-94d2-54002c9c6897 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:18.736815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e815a1d-d215-4382-94d2-54002c9c6897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:18.749937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 907/1826] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:18 2022] DELETE /identity/v3/users/546aad9b0cf5449abd8b7f1e06da8b5e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:18.778106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4e815a1d-d215-4382-94d2-54002c9c6897 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:18.779385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4e815a1d-d215-4382-94d2-54002c9c6897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:18.866942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 920/1827] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:18.890377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:18.892841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:19.007030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e815a1d-d215-4382-94d2-54002c9c6897 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:19.008478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4e815a1d-d215-4382-94d2-54002c9c6897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:19.044109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e815a1d-d215-4382-94d2-54002c9c6897 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:19.066035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:19.066745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:19.067668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:19.067962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] PATH_INFO: `/v3/users/e21010cb09c049bf87d178f95ed5f44f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:19.073954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e21010cb09c049bf87d178f95ed5f44f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:19.076761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:19.105542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af0058-7b08-4fe5-bdf6-28c605bbe7b6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:19.116549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255745d-687f-4a60-9f51-c8b4642f72ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:19.117252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255745d-687f-4a60-9f51-c8b4642f72ee service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:19.117573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255745d-687f-4a60-9f51-c8b4642f72ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:19.117844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255745d-687f-4a60-9f51-c8b4642f72ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:19.147348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255745d-687f-4a60-9f51-c8b4642f72ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:19.148834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255745d-687f-4a60-9f51-c8b4642f72ee service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:19.172843 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e21010cb09c049bf87d178f95ed5f44f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:19.174619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 908/1828] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:19.176028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681b226-6381-42b4-b347-3995bac7c40a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:19.176384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681b226-6381-42b4-b347-3995bac7c40a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:19.176908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681b226-6381-42b4-b347-3995bac7c40a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:19.187046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 909/1829] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:18:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:18:19.189315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e21010cb09c049bf87d178f95ed5f44f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:19.190291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e21010cb09c049bf87d178f95ed5f44f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:19.206357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:19.208808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e21010cb09c049bf87d178f95ed5f44f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:19.210461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-75d535b6-aeb3-43ec-a8dd-bc4ab444c328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e21010cb09c049bf87d178f95ed5f44f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:19.251054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 921/1830] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:19 2022] DELETE /identity/v3/users/e21010cb09c049bf87d178f95ed5f44f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:19.256083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4338de-813b-4121-b10b-7b8523c2173e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:19.256467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4338de-813b-4121-b10b-7b8523c2173e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:19.256758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4338de-813b-4121-b10b-7b8523c2173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:19.275640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4833-5399-4f68-9d9a-a8732f8c1da8 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:19.275969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4833-5399-4f68-9d9a-a8732f8c1da8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:19.276198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4833-5399-4f68-9d9a-a8732f8c1da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:19.303146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6f4338de-813b-4121-b10b-7b8523c2173e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:19.305381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6f4338de-813b-4121-b10b-7b8523c2173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:19.331831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e54f4833-5399-4f68-9d9a-a8732f8c1da8 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:19.335452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e54f4833-5399-4f68-9d9a-a8732f8c1da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:19.355384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:19.445940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 910/1831] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:19.487763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4338de-813b-4121-b10b-7b8523c2173e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:19.491826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6f4338de-813b-4121-b10b-7b8523c2173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:19.534110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 922/1832] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:18:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:19.601013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4338de-813b-4121-b10b-7b8523c2173e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:19.601013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:19.601013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:19.601013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:19.601361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:19.610356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:19.612335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:19.624615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 911/1833] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:20.256811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f4833-5399-4f68-9d9a-a8732f8c1da8 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:20.259625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e54f4833-5399-4f68-9d9a-a8732f8c1da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:20.279110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f4833-5399-4f68-9d9a-a8732f8c1da8 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:20.294684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eb2f9-f183-4c41-9876-d18fa0c6da1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:20.296136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb2f9-f183-4c41-9876-d18fa0c6da1d service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:20.296655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb2f9-f183-4c41-9876-d18fa0c6da1d service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:20.297127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb2f9-f183-4c41-9876-d18fa0c6da1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:20.316205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713eb2f9-f183-4c41-9876-d18fa0c6da1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:20.320093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713eb2f9-f183-4c41-9876-d18fa0c6da1d service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:20.335628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 923/1834] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:20.411260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:20.412895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:20.427186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43306570-8934-496e-9ecd-2c1d5b63a3ce service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:20.434116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:20.434768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:20.435049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:20.435320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] PATH_INFO: `/v3/projects/b998ed7178a7459db8ccbf9c38877497` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:20.436460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b998ed7178a7459db8ccbf9c38877497)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:20.438431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:20.499951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b998ed7178a7459db8ccbf9c38877497', 'request_id': 'req-19b57cac-60b8-4b88-937f-a1c775e1173a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.510501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b998ed7178a7459db8ccbf9c38877497', 'request_id': 'req-19b57cac-60b8-4b88-937f-a1c775e1173a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.518279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b998ed7178a7459db8ccbf9c38877497', 'request_id': 'req-19b57cac-60b8-4b88-937f-a1c775e1173a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.530808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] The token cache is being invalidate because project b998ed7178a7459db8ccbf9c38877497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:20.531180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.531939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.533528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 912/1835] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:20 2022] DELETE /identity/v3/projects/b998ed7178a7459db8ccbf9c38877497 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:20.538377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eb2f9-f183-4c41-9876-d18fa0c6da1d service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:20.549618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eb2f9-f183-4c41-9876-d18fa0c6da1d service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:20.611346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:20.612228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:20.612748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:20.613298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] PATH_INFO: `/v3/users/bbd464bc69354cd08db08049e69750ee` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:20.615383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd464bc69354cd08db08049e69750ee)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:20.617468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:20.710909 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd464bc69354cd08db08049e69750ee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.720098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd464bc69354cd08db08049e69750ee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.721482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd464bc69354cd08db08049e69750ee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.743468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:20.746619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd464bc69354cd08db08049e69750ee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.747667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd464bc69354cd08db08049e69750ee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:20.778987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 924/1836] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:20 2022] DELETE /identity/v3/users/bbd464bc69354cd08db08049e69750ee => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:21.166701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.168355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:21.178098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.179739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:21.203097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b57cac-60b8-4b88-937f-a1c775e1173a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.266005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:21.278204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.278982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.279680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.279874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] PATH_INFO: `/v3/projects/87fb4e50ed144393badee4cacc838f54` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.282957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87fb4e50ed144393badee4cacc838f54)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.284832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7e22f-3f73-4ff0-9bb9-9d8d4db2e781 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.289835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.295559 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.297037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.297733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.298018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] PATH_INFO: `/v3/projects/201840de51b645e7afa0516404e61232` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.299189 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201840de51b645e7afa0516404e61232)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.302365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.353962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87fb4e50ed144393badee4cacc838f54', 'request_id': 'req-54b97dad-7855-415e-bff6-e09ef7851659'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.355420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201840de51b645e7afa0516404e61232', 'request_id': 'req-70bd0824-a101-4184-bc37-9123aafb16a8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.362394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87fb4e50ed144393badee4cacc838f54', 'request_id': 'req-54b97dad-7855-415e-bff6-e09ef7851659'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.364669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201840de51b645e7afa0516404e61232', 'request_id': 'req-70bd0824-a101-4184-bc37-9123aafb16a8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.370988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87fb4e50ed144393badee4cacc838f54', 'request_id': 'req-54b97dad-7855-415e-bff6-e09ef7851659'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.371887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201840de51b645e7afa0516404e61232', 'request_id': 'req-70bd0824-a101-4184-bc37-9123aafb16a8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.385736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] The token cache is being invalidate because project 87fb4e50ed144393badee4cacc838f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:21.386029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.388019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-54b97dad-7855-415e-bff6-e09ef7851659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.391426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] The token cache is being invalidate because project 201840de51b645e7afa0516404e61232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:21.391728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.392604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 913/1837] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:21 2022] DELETE /identity/v3/projects/87fb4e50ed144393badee4cacc838f54 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:21.392757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.395618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 925/1838] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:21 2022] DELETE /identity/v3/projects/201840de51b645e7afa0516404e61232 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:21.397641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869812-9cef-4c50-8f90-86b739258d33 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.397927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869812-9cef-4c50-8f90-86b739258d33 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.398199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869812-9cef-4c50-8f90-86b739258d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.399666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.406331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:21.422583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fe869812-9cef-4c50-8f90-86b739258d33 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:21.423806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fe869812-9cef-4c50-8f90-86b739258d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:21.489520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd0824-a101-4184-bc37-9123aafb16a8 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.490666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 914/1839] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:21.497167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.498623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.499103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.499382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] PATH_INFO: `/v3/users/5dd4f5e098594e7eb99e4a8c8551847a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.500496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecfcfe-2cef-4b27-a95b-8f6df7c705a1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.500977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd4f5e098594e7eb99e4a8c8551847a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.501073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecfcfe-2cef-4b27-a95b-8f6df7c705a1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.501408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecfcfe-2cef-4b27-a95b-8f6df7c705a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.502757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.528068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-18ecfcfe-2cef-4b27-a95b-8f6df7c705a1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:21.529574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-18ecfcfe-2cef-4b27-a95b-8f6df7c705a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:21.530898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:21.540948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 915/1840] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:21.545737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecfcfe-2cef-4b27-a95b-8f6df7c705a1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.559779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecfcfe-2cef-4b27-a95b-8f6df7c705a1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.565389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.566004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.566275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.566546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.567254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd4f5e098594e7eb99e4a8c8551847a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.567481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:21.567847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.569774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.575645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd4f5e098594e7eb99e4a8c8551847a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.576355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd4f5e098594e7eb99e4a8c8551847a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.590168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 916/1841] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:21.592591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd4f5e098594e7eb99e4a8c8551847a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.593310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd4f5e098594e7eb99e4a8c8551847a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:21.594200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.596693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:21.607336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 926/1842] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:21 2022] DELETE /identity/v3/users/5dd4f5e098594e7eb99e4a8c8551847a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:21.610660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ba04f-85ab-4796-9f20-8a18188115a7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.678266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610557e-d200-46b5-be11-b7719a382bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.679001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610557e-d200-46b5-be11-b7719a382bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.679380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610557e-d200-46b5-be11-b7719a382bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.679738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610557e-d200-46b5-be11-b7719a382bfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.680603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610557e-d200-46b5-be11-b7719a382bfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.682490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610557e-d200-46b5-be11-b7719a382bfb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.693143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:21.721108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 917/1843] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:21 2022] POST /identity/v3/projects => generated 399 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:18:21.725464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.737297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c48c6-70ff-45f6-8ab1-1b90aaeee0a4 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.817060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c45be-66f8-4613-9d99-05ac1a9d665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.817706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c45be-66f8-4613-9d99-05ac1a9d665f admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.817985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c45be-66f8-4613-9d99-05ac1a9d665f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.818255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c45be-66f8-4613-9d99-05ac1a9d665f admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.819388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c45be-66f8-4613-9d99-05ac1a9d665f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.821341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c45be-66f8-4613-9d99-05ac1a9d665f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.843108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 927/1844] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:21 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:21.847427 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610557e-d200-46b5-be11-b7719a382bfb admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.860233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610557e-d200-46b5-be11-b7719a382bfb admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.867206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9080e5-16d9-43b2-a0b2-8f26ed21fe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.868515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9080e5-16d9-43b2-a0b2-8f26ed21fe73 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.868795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9080e5-16d9-43b2-a0b2-8f26ed21fe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.869425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9080e5-16d9-43b2-a0b2-8f26ed21fe73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.870247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9080e5-16d9-43b2-a0b2-8f26ed21fe73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.872295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9080e5-16d9-43b2-a0b2-8f26ed21fe73 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.887707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 918/1845] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:21.893245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c45be-66f8-4613-9d99-05ac1a9d665f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.905456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c45be-66f8-4613-9d99-05ac1a9d665f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.911490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.912399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.912722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.912998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.913651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.915529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.927195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 928/1846] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:21.932686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9080e5-16d9-43b2-a0b2-8f26ed21fe73 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.947385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9080e5-16d9-43b2-a0b2-8f26ed21fe73 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.952185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:21.954864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:21.963450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4050275-a463-42cc-9dc2-e0fa9d3e5955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.964141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4050275-a463-42cc-9dc2-e0fa9d3e5955 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.964424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4050275-a463-42cc-9dc2-e0fa9d3e5955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.964739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4050275-a463-42cc-9dc2-e0fa9d3e5955 admin admin] PATH_INFO: `/v3/projects/6ea37130fa0f44918cfd4523eb445a36/users/22b9a92c31c64eccad61204522a1ea53/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.972247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736243a-131e-44fb-ac4b-35a61fd0505c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:21.979682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:21.980335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:21.980644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:21.980923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] PATH_INFO: `/v3/projects/07aab95d66a04bd3ad1cf29ebbc6624a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:21.990867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=07aab95d66a04bd3ad1cf29ebbc6624a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.992526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4050275-a463-42cc-9dc2-e0fa9d3e5955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea37130fa0f44918cfd4523eb445a36, user_id=22b9a92c31c64eccad61204522a1ea53, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:21.993642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:21.995315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4050275-a463-42cc-9dc2-e0fa9d3e5955 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.014657 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 919/1847] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:21 2022] PUT /identity/v3/projects/6ea37130fa0f44918cfd4523eb445a36/users/22b9a92c31c64eccad61204522a1ea53/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:22.028130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380ec6f-180f-4738-ace7-64b5f4905ee0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.028541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380ec6f-180f-4738-ace7-64b5f4905ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.028782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380ec6f-180f-4738-ace7-64b5f4905ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.067466 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7380ec6f-180f-4738-ace7-64b5f4905ee0 None None] MFA Rules not processed for user `22b9a92c31c64eccad61204522a1ea53`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:22.069476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7380ec6f-180f-4738-ace7-64b5f4905ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:22.069703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07aab95d66a04bd3ad1cf29ebbc6624a', 'request_id': 'req-67450133-2081-4c1a-8382-d0aed6cbeffe'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:22.077774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07aab95d66a04bd3ad1cf29ebbc6624a', 'request_id': 'req-67450133-2081-4c1a-8382-d0aed6cbeffe'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:22.086919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07aab95d66a04bd3ad1cf29ebbc6624a', 'request_id': 'req-67450133-2081-4c1a-8382-d0aed6cbeffe'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:22.117015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=15,keystone:SELECT=80,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:22.121505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] The token cache is being invalidate because project 07aab95d66a04bd3ad1cf29ebbc6624a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:22.121981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:22.127903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-67450133-2081-4c1a-8382-d0aed6cbeffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:22.130724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 929/1848] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:21 2022] DELETE /identity/v3/projects/07aab95d66a04bd3ad1cf29ebbc6624a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:22.138210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d996-2903-45f0-aad1-d4752a45f824 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.138549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d996-2903-45f0-aad1-d4752a45f824 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.138833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d996-2903-45f0-aad1-d4752a45f824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.166320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-2637d996-2903-45f0-aad1-d4752a45f824 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:22.167650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2637d996-2903-45f0-aad1-d4752a45f824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:22.181228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 920/1849] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:22.185351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380ec6f-180f-4738-ace7-64b5f4905ee0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.187646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7380ec6f-180f-4738-ace7-64b5f4905ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:22.255064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 930/1850] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:22.265689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920aa2-c23d-42d6-addb-804b4e30cd51 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.266399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920aa2-c23d-42d6-addb-804b4e30cd51 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.266878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920aa2-c23d-42d6-addb-804b4e30cd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.286786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380ec6f-180f-4738-ace7-64b5f4905ee0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.294305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-03920aa2-c23d-42d6-addb-804b4e30cd51 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:22.295077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0325d7-c636-401b-a861-289cc57c10df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.295835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0325d7-c636-401b-a861-289cc57c10df admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.296287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0325d7-c636-401b-a861-289cc57c10df admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.297112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0325d7-c636-401b-a861-289cc57c10df admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.297547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0325d7-c636-401b-a861-289cc57c10df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.298013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-03920aa2-c23d-42d6-addb-804b4e30cd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:22.302730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0325d7-c636-401b-a861-289cc57c10df admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.309688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 931/1851] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:22.313763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03920aa2-c23d-42d6-addb-804b4e30cd51 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.329286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03920aa2-c23d-42d6-addb-804b4e30cd51 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.336174 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.341815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.342360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.342738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.343577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:22.344053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.347862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.350506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 921/1852] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:22 2022] POST /identity/v3/projects => generated 399 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:22.354301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0325d7-c636-401b-a861-289cc57c10df admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.365873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 932/1853] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:22.370529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.383226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d78e1-34df-45c9-b8f8-03277b0cfd49 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.393575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:22.459949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0325d7-c636-401b-a861-289cc57c10df admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.470483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237b9bc-396b-4edd-8b46-525400a54115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.471074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237b9bc-396b-4edd-8b46-525400a54115 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.471615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237b9bc-396b-4edd-8b46-525400a54115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.471947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237b9bc-396b-4edd-8b46-525400a54115 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.473681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237b9bc-396b-4edd-8b46-525400a54115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.477028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237b9bc-396b-4edd-8b46-525400a54115 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.494842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f05083-dea9-4e5d-a6dc-8d80dcde9093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.496930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f05083-dea9-4e5d-a6dc-8d80dcde9093 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.497586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f05083-dea9-4e5d-a6dc-8d80dcde9093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.499968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f05083-dea9-4e5d-a6dc-8d80dcde9093 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.500785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f05083-dea9-4e5d-a6dc-8d80dcde9093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.503845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f05083-dea9-4e5d-a6dc-8d80dcde9093 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.521930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 933/1854] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:22 2022] POST /identity/v3/projects => generated 361 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:22.526537 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237b9bc-396b-4edd-8b46-525400a54115 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.535493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 922/1855] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:22 2022] POST /identity/v3/users => generated 306 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:22.540037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f05083-dea9-4e5d-a6dc-8d80dcde9093 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.554398 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f05083-dea9-4e5d-a6dc-8d80dcde9093 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.616878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237b9bc-396b-4edd-8b46-525400a54115 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.623730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ebf4e-1735-4e50-b90b-ea1441e401f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.624425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ebf4e-1735-4e50-b90b-ea1441e401f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.624803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ebf4e-1735-4e50-b90b-ea1441e401f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.625298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ebf4e-1735-4e50-b90b-ea1441e401f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.625840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ebf4e-1735-4e50-b90b-ea1441e401f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.628763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ebf4e-1735-4e50-b90b-ea1441e401f2 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.640417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:22.659273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 934/1856] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:22 2022] POST /identity/v3/users => generated 281 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:22.665915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ebf4e-1735-4e50-b90b-ea1441e401f2 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.692283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ebf4e-1735-4e50-b90b-ea1441e401f2 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.703914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd0f5-9327-4c29-a505-0e8c88182370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.704778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd0f5-9327-4c29-a505-0e8c88182370 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.705059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd0f5-9327-4c29-a505-0e8c88182370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.705329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd0f5-9327-4c29-a505-0e8c88182370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.707321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bd0f5-9327-4c29-a505-0e8c88182370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.711331 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bd0f5-9327-4c29-a505-0e8c88182370 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.724461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 935/1857] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:22.728634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd0f5-9327-4c29-a505-0e8c88182370 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.733629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a2a9c-fdee-4dd3-8e0a-031b7fa9e453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.734596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a2a9c-fdee-4dd3-8e0a-031b7fa9e453 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.735602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a2a9c-fdee-4dd3-8e0a-031b7fa9e453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.736106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a2a9c-fdee-4dd3-8e0a-031b7fa9e453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.737283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a2a9c-fdee-4dd3-8e0a-031b7fa9e453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.738882 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd0f5-9327-4c29-a505-0e8c88182370 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.741346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a2a9c-fdee-4dd3-8e0a-031b7fa9e453 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.744663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f9c91-610a-4141-9ba8-349054b5f437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.745543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f9c91-610a-4141-9ba8-349054b5f437 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.745669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f9c91-610a-4141-9ba8-349054b5f437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.745929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f9c91-610a-4141-9ba8-349054b5f437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.746580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f9c91-610a-4141-9ba8-349054b5f437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.748210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f9c91-610a-4141-9ba8-349054b5f437 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.755829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 923/1858] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:18:22.759581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a2a9c-fdee-4dd3-8e0a-031b7fa9e453 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.769733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 936/1859] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:22.771139 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f9c91-610a-4141-9ba8-349054b5f437 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.775949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a2a9c-fdee-4dd3-8e0a-031b7fa9e453 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.781296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cab15-58ff-4a06-a900-9feba4d72864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.782100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cab15-58ff-4a06-a900-9feba4d72864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.782386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cab15-58ff-4a06-a900-9feba4d72864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.782486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f9c91-610a-4141-9ba8-349054b5f437 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.782661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cab15-58ff-4a06-a900-9feba4d72864 admin admin] PATH_INFO: `/v3/projects/a4b1bf4b145f450484db875c786c1147/users/8a62d79ddd9740fdb8fd61f273ecf037/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.788441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3703f2-6da7-4243-940d-24f9a17f10f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:22.789369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3703f2-6da7-4243-940d-24f9a17f10f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.789790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3703f2-6da7-4243-940d-24f9a17f10f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.790066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3703f2-6da7-4243-940d-24f9a17f10f8 admin admin] PATH_INFO: `/v3/projects/56de1d9902f04a9c8ae28836a154b155/users/76d4a80ce575486ebeffb64e16a3688e/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.803522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cab15-58ff-4a06-a900-9feba4d72864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b1bf4b145f450484db875c786c1147, user_id=8a62d79ddd9740fdb8fd61f273ecf037, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.805649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cab15-58ff-4a06-a900-9feba4d72864 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.813310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3703f2-6da7-4243-940d-24f9a17f10f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56de1d9902f04a9c8ae28836a154b155, user_id=76d4a80ce575486ebeffb64e16a3688e, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:22.816594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3703f2-6da7-4243-940d-24f9a17f10f8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:22.827339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 924/1860] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:22 2022] PUT /identity/v3/projects/a4b1bf4b145f450484db875c786c1147/users/8a62d79ddd9740fdb8fd61f273ecf037/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:22.829446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cab15-58ff-4a06-a900-9feba4d72864 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:22.841794 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 937/1861] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:22 2022] PUT /identity/v3/projects/56de1d9902f04a9c8ae28836a154b155/users/76d4a80ce575486ebeffb64e16a3688e/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:22.853257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b1e1-f6ec-4c56-a3bb-5411e4ad81eb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:22.855012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b1e1-f6ec-4c56-a3bb-5411e4ad81eb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:22.855482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b1e1-f6ec-4c56-a3bb-5411e4ad81eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:22.888858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6136b1e1-f6ec-4c56-a3bb-5411e4ad81eb None None] MFA Rules not processed for user `76d4a80ce575486ebeffb64e16a3688e`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:22.890852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6136b1e1-f6ec-4c56-a3bb-5411e4ad81eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:22.912214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cab15-58ff-4a06-a900-9feba4d72864 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:22.955767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:22.966273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 938/1862] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:23.004222 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8d64c-d3cd-40e3-9d3a-b2719b4c2915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:23.005401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8d64c-d3cd-40e3-9d3a-b2719b4c2915 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.005898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8d64c-d3cd-40e3-9d3a-b2719b4c2915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.006380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8d64c-d3cd-40e3-9d3a-b2719b4c2915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.007386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8d64c-d3cd-40e3-9d3a-b2719b4c2915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:23.009875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8d64c-d3cd-40e3-9d3a-b2719b4c2915 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:23.028816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136b1e1-f6ec-4c56-a3bb-5411e4ad81eb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:23.030321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6136b1e1-f6ec-4c56-a3bb-5411e4ad81eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:23.036647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 925/1863] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:23.037667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8d64c-d3cd-40e3-9d3a-b2719b4c2915 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:23.053941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8d64c-d3cd-40e3-9d3a-b2719b4c2915 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:23.059629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f63060-0344-4ee0-ad15-7a323cf2dc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:23.060447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63060-0344-4ee0-ad15-7a323cf2dc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.060823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63060-0344-4ee0-ad15-7a323cf2dc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.061255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63060-0344-4ee0-ad15-7a323cf2dc10 admin admin] PATH_INFO: `/v3/projects/a4b1bf4b145f450484db875c786c1147/users/8a62d79ddd9740fdb8fd61f273ecf037/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.063418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f63060-0344-4ee0-ad15-7a323cf2dc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b1bf4b145f450484db875c786c1147, user_id=8a62d79ddd9740fdb8fd61f273ecf037, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:23.068799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f63060-0344-4ee0-ad15-7a323cf2dc10 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:23.080174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 926/1864] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:23 2022] PUT /identity/v3/projects/a4b1bf4b145f450484db875c786c1147/users/8a62d79ddd9740fdb8fd61f273ecf037/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:23.091423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81bed3-6e37-47d8-b65c-4a503eb1099f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.091714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81bed3-6e37-47d8-b65c-4a503eb1099f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.091993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81bed3-6e37-47d8-b65c-4a503eb1099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.114987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136b1e1-f6ec-4c56-a3bb-5411e4ad81eb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:23.128818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4a81bed3-6e37-47d8-b65c-4a503eb1099f None None] MFA Rules not processed for user `8a62d79ddd9740fdb8fd61f273ecf037`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:23.131257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4a81bed3-6e37-47d8-b65c-4a503eb1099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:23.184286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0411e0-4580-48ac-9a50-69dc57a474c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:23.185072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0411e0-4580-48ac-9a50-69dc57a474c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.185354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0411e0-4580-48ac-9a50-69dc57a474c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.185625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0411e0-4580-48ac-9a50-69dc57a474c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.191779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:23.232670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 927/1865] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:23 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:23.250690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81bed3-6e37-47d8-b65c-4a503eb1099f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:23.253147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4a81bed3-6e37-47d8-b65c-4a503eb1099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:23.276689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0411e0-4580-48ac-9a50-69dc57a474c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:23.278521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0411e0-4580-48ac-9a50-69dc57a474c6 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:23.290271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 939/1866] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:23.305622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81bed3-6e37-47d8-b65c-4a503eb1099f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:23.311816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c601-3678-4d2d-8926-08b0f91f3519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:23.312378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0411e0-4580-48ac-9a50-69dc57a474c6 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:23.313017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c601-3678-4d2d-8926-08b0f91f3519 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.313017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c601-3678-4d2d-8926-08b0f91f3519 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.313017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340c601-3678-4d2d-8926-08b0f91f3519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.323988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340c601-3678-4d2d-8926-08b0f91f3519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:23.325702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340c601-3678-4d2d-8926-08b0f91f3519 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:23.336179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 928/1867] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:23 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:23.393325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0411e0-4580-48ac-9a50-69dc57a474c6 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:23.399060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:23.399693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.399974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.400246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] PATH_INFO: `/v3/users/76d4a80ce575486ebeffb64e16a3688e` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.401636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d4a80ce575486ebeffb64e16a3688e)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:23.403556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:23.456497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c601-3678-4d2d-8926-08b0f91f3519 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:23.465301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d4a80ce575486ebeffb64e16a3688e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:23.467537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340c601-3678-4d2d-8926-08b0f91f3519 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:23.471374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:23.472234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.472593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d4a80ce575486ebeffb64e16a3688e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:23.472699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.473012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.473839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d4a80ce575486ebeffb64e16a3688e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:23.477022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:23.488299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d4a80ce575486ebeffb64e16a3688e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:23.490077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9031c242-3793-4dc0-85d2-6a06ad3ea726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d4a80ce575486ebeffb64e16a3688e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:23.507445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 940/1868] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:23 2022] DELETE /identity/v3/users/76d4a80ce575486ebeffb64e16a3688e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:23.511858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063917db-2a66-4b6a-8f84-435243dc6fb4 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.512140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063917db-2a66-4b6a-8f84-435243dc6fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.512506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063917db-2a66-4b6a-8f84-435243dc6fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.533421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-063917db-2a66-4b6a-8f84-435243dc6fb4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:23.535200 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-063917db-2a66-4b6a-8f84-435243dc6fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:23.547472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:23.549288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:23.560392 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 929/1869] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:23.573630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:23.622310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 941/1870] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:23.632036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:23.633638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:23.672374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:23.704517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759181c-2cf9-4705-8e02-75ddacd0a606 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:23.783830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2277a-d3f8-49b7-bb9c-aec1810edb5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:23.784278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2277a-d3f8-49b7-bb9c-aec1810edb5e service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:23.784684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2277a-d3f8-49b7-bb9c-aec1810edb5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:23.785095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2277a-d3f8-49b7-bb9c-aec1810edb5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:23.797278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2277a-d3f8-49b7-bb9c-aec1810edb5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:23.799473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2277a-d3f8-49b7-bb9c-aec1810edb5e service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:23.828003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 930/1871] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:24.683279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063917db-2a66-4b6a-8f84-435243dc6fb4 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:24.684759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-063917db-2a66-4b6a-8f84-435243dc6fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:24.699311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063917db-2a66-4b6a-8f84-435243dc6fb4 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:24.704196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:24.704862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:24.705139 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:24.705517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] PATH_INFO: `/v3/projects/56de1d9902f04a9c8ae28836a154b155` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:24.707073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56de1d9902f04a9c8ae28836a154b155)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:24.711360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:24.776696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56de1d9902f04a9c8ae28836a154b155', 'request_id': 'req-f7ac4bac-db72-4a2b-a483-3eaecf841f29'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:24.787814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56de1d9902f04a9c8ae28836a154b155', 'request_id': 'req-f7ac4bac-db72-4a2b-a483-3eaecf841f29'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:24.805170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56de1d9902f04a9c8ae28836a154b155', 'request_id': 'req-f7ac4bac-db72-4a2b-a483-3eaecf841f29'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:24.820317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] The token cache is being invalidate because project 56de1d9902f04a9c8ae28836a154b155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:24.820780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:24.821525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f7ac4bac-db72-4a2b-a483-3eaecf841f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:24.823388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 942/1872] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:24 2022] DELETE /identity/v3/projects/56de1d9902f04a9c8ae28836a154b155 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:24.828302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8133e-80c3-443b-a003-accec8e528fb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:24.828939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8133e-80c3-443b-a003-accec8e528fb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:24.828939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8133e-80c3-443b-a003-accec8e528fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:24.850530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-71f8133e-80c3-443b-a003-accec8e528fb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:24.851753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-71f8133e-80c3-443b-a003-accec8e528fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:24.922407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 931/1873] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:24.930310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8e87-b49c-444e-ba9d-06cba44a9658 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:24.930595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8e87-b49c-444e-ba9d-06cba44a9658 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:24.930867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8e87-b49c-444e-ba9d-06cba44a9658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:24.954965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-333d8e87-b49c-444e-ba9d-06cba44a9658 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:24.956303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-333d8e87-b49c-444e-ba9d-06cba44a9658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:24.965384 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 943/1874] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:24.970675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8133e-80c3-443b-a003-accec8e528fb None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:24.985529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8133e-80c3-443b-a003-accec8e528fb None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:24.991270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:24.992186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:24.992780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:24.993035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:24.993774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:24.994177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:24.996002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:25.020673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 932/1875] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:25.026214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d8e87-b49c-444e-ba9d-06cba44a9658 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:25.041387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d8e87-b49c-444e-ba9d-06cba44a9658 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:25.048906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acc3e9-58ac-4397-bc03-0e25034498aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:25.049767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acc3e9-58ac-4397-bc03-0e25034498aa admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.050052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acc3e9-58ac-4397-bc03-0e25034498aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.050323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acc3e9-58ac-4397-bc03-0e25034498aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.051112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41acc3e9-58ac-4397-bc03-0e25034498aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:25.053247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41acc3e9-58ac-4397-bc03-0e25034498aa admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:25.104787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 944/1876] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:25 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:25.109640 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:25.119893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e9961-76c5-43c8-8092-2595c3d03592 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:25.224781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97217eba-97e0-4169-b8e9-475ed9fafc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:25.226404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97217eba-97e0-4169-b8e9-475ed9fafc68 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.227394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97217eba-97e0-4169-b8e9-475ed9fafc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.227958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97217eba-97e0-4169-b8e9-475ed9fafc68 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.228811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97217eba-97e0-4169-b8e9-475ed9fafc68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:25.230928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97217eba-97e0-4169-b8e9-475ed9fafc68 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:25.263570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 933/1877] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:25 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:18:25.268671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acc3e9-58ac-4397-bc03-0e25034498aa admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:25.302088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acc3e9-58ac-4397-bc03-0e25034498aa admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:25.308486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc95895-5867-4e1a-b650-3f73c5370ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:25.309170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc95895-5867-4e1a-b650-3f73c5370ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.309451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc95895-5867-4e1a-b650-3f73c5370ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.309726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc95895-5867-4e1a-b650-3f73c5370ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.310393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc95895-5867-4e1a-b650-3f73c5370ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:25.312011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc95895-5867-4e1a-b650-3f73c5370ac5 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:25.323261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 945/1878] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:25.327471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97217eba-97e0-4169-b8e9-475ed9fafc68 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:25.342845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97217eba-97e0-4169-b8e9-475ed9fafc68 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:25.356717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fab179-a262-4cb7-a50a-3d25ba198644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:25.357228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fab179-a262-4cb7-a50a-3d25ba198644 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.357514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fab179-a262-4cb7-a50a-3d25ba198644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.357783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fab179-a262-4cb7-a50a-3d25ba198644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.358428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fab179-a262-4cb7-a50a-3d25ba198644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:25.362923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fab179-a262-4cb7-a50a-3d25ba198644 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:25.365067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:25.385212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 934/1879] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:25.389347 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc95895-5867-4e1a-b650-3f73c5370ac5 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:25.405390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc95895-5867-4e1a-b650-3f73c5370ac5 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:25.413136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bfdc-2608-4376-80dd-70fc6c940165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:25.414005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bfdc-2608-4376-80dd-70fc6c940165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.414556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bfdc-2608-4376-80dd-70fc6c940165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.414830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945bfdc-2608-4376-80dd-70fc6c940165 admin admin] PATH_INFO: `/v3/projects/fef3ab8cfef54a8aa47767dd254a2c0c/users/9f81a268eebc4ba88cbb75fe94f5ec95/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.436125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945bfdc-2608-4376-80dd-70fc6c940165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef3ab8cfef54a8aa47767dd254a2c0c, user_id=9f81a268eebc4ba88cbb75fe94f5ec95, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:25.442296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945bfdc-2608-4376-80dd-70fc6c940165 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:25.466639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 946/1880] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:25 2022] PUT /identity/v3/projects/fef3ab8cfef54a8aa47767dd254a2c0c/users/9f81a268eebc4ba88cbb75fe94f5ec95/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:25.477678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed87fa6-db11-4124-80d6-6da3b0f6fa0b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.478116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed87fa6-db11-4124-80d6-6da3b0f6fa0b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.478482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed87fa6-db11-4124-80d6-6da3b0f6fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.530660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6ed87fa6-db11-4124-80d6-6da3b0f6fa0b None None] MFA Rules not processed for user `9f81a268eebc4ba88cbb75fe94f5ec95`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:25.532393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6ed87fa6-db11-4124-80d6-6da3b0f6fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:25.667646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 935/1881] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:25.673115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bfdc-2608-4376-80dd-70fc6c940165 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:25.742854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:25.786463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945bfdc-2608-4376-80dd-70fc6c940165 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:25.801360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca004129-bd7e-45d3-b5dd-5ead42ae1856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:25.802157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca004129-bd7e-45d3-b5dd-5ead42ae1856 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.802532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca004129-bd7e-45d3-b5dd-5ead42ae1856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.802998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca004129-bd7e-45d3-b5dd-5ead42ae1856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.803782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca004129-bd7e-45d3-b5dd-5ead42ae1856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:25.805891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca004129-bd7e-45d3-b5dd-5ead42ae1856 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:25.842391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 947/1882] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:25 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:25.843703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed87fa6-db11-4124-80d6-6da3b0f6fa0b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:25.857672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed87fa6-db11-4124-80d6-6da3b0f6fa0b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:25.930730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a58a-8303-4186-9388-ae988ad7c18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:25.931453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a58a-8303-4186-9388-ae988ad7c18b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.931801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a58a-8303-4186-9388-ae988ad7c18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.932150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a58a-8303-4186-9388-ae988ad7c18b admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.932899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6a58a-8303-4186-9388-ae988ad7c18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:25.935112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6a58a-8303-4186-9388-ae988ad7c18b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:25.972012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 936/1883] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:25 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:25.973904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca004129-bd7e-45d3-b5dd-5ead42ae1856 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:25.990425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca004129-bd7e-45d3-b5dd-5ead42ae1856 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:25.997576 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6c8d3-15e6-4983-a1d8-253f2191f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:25.998313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6c8d3-15e6-4983-a1d8-253f2191f960 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:25.998771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6c8d3-15e6-4983-a1d8-253f2191f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:25.999049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6c8d3-15e6-4983-a1d8-253f2191f960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:25.999702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6c8d3-15e6-4983-a1d8-253f2191f960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:26.002573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6c8d3-15e6-4983-a1d8-253f2191f960 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:26.020609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 948/1884] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:18:26.023215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a58a-8303-4186-9388-ae988ad7c18b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:26.034567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a58a-8303-4186-9388-ae988ad7c18b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:26.040977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663987d-a3f5-4faa-96fe-88fc6a5aa571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:26.041660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663987d-a3f5-4faa-96fe-88fc6a5aa571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:26.041942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663987d-a3f5-4faa-96fe-88fc6a5aa571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:26.042223 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663987d-a3f5-4faa-96fe-88fc6a5aa571 admin admin] PATH_INFO: `/v3/projects/17188ef625ce46c5ad9215d991c646e6/users/f54904f099d84d7991ba2a74dda9f54c/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:26.072815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663987d-a3f5-4faa-96fe-88fc6a5aa571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17188ef625ce46c5ad9215d991c646e6, user_id=f54904f099d84d7991ba2a74dda9f54c, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:26.076358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663987d-a3f5-4faa-96fe-88fc6a5aa571 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:26.109033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 937/1885] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:26 2022] PUT /identity/v3/projects/17188ef625ce46c5ad9215d991c646e6/users/f54904f099d84d7991ba2a74dda9f54c/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:26.109515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6c8d3-15e6-4983-a1d8-253f2191f960 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:26.125748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6c8d3-15e6-4983-a1d8-253f2191f960 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:26.185204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96645b6b-ab63-41e2-aae7-a81fcc877f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:26.186145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96645b6b-ab63-41e2-aae7-a81fcc877f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:26.186419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96645b6b-ab63-41e2-aae7-a81fcc877f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:26.186689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96645b6b-ab63-41e2-aae7-a81fcc877f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:26.187337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96645b6b-ab63-41e2-aae7-a81fcc877f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:26.189070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96645b6b-ab63-41e2-aae7-a81fcc877f90 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:26.200895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 949/1886] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:18:26.204976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663987d-a3f5-4faa-96fe-88fc6a5aa571 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:26.215519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663987d-a3f5-4faa-96fe-88fc6a5aa571 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:26.220362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:26.221067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:26.221344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:26.221621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] PATH_INFO: `/v3/projects/17188ef625ce46c5ad9215d991c646e6/users/f54904f099d84d7991ba2a74dda9f54c/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:26.223661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17188ef625ce46c5ad9215d991c646e6, user_id=f54904f099d84d7991ba2a74dda9f54c, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:26.226457 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:26.237700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 938/1887] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:26 2022] PUT /identity/v3/projects/17188ef625ce46c5ad9215d991c646e6/users/f54904f099d84d7991ba2a74dda9f54c/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:26.248909 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4985-517d-48e6-9973-634fd7f1680c None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:26.249295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4985-517d-48e6-9973-634fd7f1680c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:26.249855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4985-517d-48e6-9973-634fd7f1680c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:26.281570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f5fe4985-517d-48e6-9973-634fd7f1680c None None] MFA Rules not processed for user `f54904f099d84d7991ba2a74dda9f54c`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:26.282822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f5fe4985-517d-48e6-9973-634fd7f1680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:26.359876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 950/1888] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:26.371395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:26.373150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:26.447105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:26.468155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac88f9a-a8ae-4d5f-bb48-4c8b1dc21ba6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:26.481313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a947b-a664-45fb-83ab-efdb917bd1c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:26.482262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a947b-a664-45fb-83ab-efdb917bd1c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:26.482705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a947b-a664-45fb-83ab-efdb917bd1c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:26.483157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a947b-a664-45fb-83ab-efdb917bd1c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:26.559947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a947b-a664-45fb-83ab-efdb917bd1c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:26.561432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a947b-a664-45fb-83ab-efdb917bd1c1 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:26.572240 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 939/1889] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:28.504235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe4985-517d-48e6-9973-634fd7f1680c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:28.505763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f5fe4985-517d-48e6-9973-634fd7f1680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:28.564523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:28.588992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe4985-517d-48e6-9973-634fd7f1680c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:28.594096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:28.594802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:28.595689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:28.595689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] PATH_INFO: `/v3/users/22b9a92c31c64eccad61204522a1ea53` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:28.597060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b9a92c31c64eccad61204522a1ea53)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:28.599849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:28.717777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b9a92c31c64eccad61204522a1ea53'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:28.727877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b9a92c31c64eccad61204522a1ea53'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:28.729878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b9a92c31c64eccad61204522a1ea53'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:28.746953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b9a92c31c64eccad61204522a1ea53'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:28.747751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b9a92c31c64eccad61204522a1ea53'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:28.765574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 951/1890] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:28 2022] DELETE /identity/v3/users/22b9a92c31c64eccad61204522a1ea53 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:28.772941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1227efa-73bd-4c44-a75a-4e4847081dde None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:28.773468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1227efa-73bd-4c44-a75a-4e4847081dde None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:28.773853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1227efa-73bd-4c44-a75a-4e4847081dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:28.798310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e1227efa-73bd-4c44-a75a-4e4847081dde None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:28.800243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e1227efa-73bd-4c44-a75a-4e4847081dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:28.928634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 940/1891] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:28.944813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:28.946313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:28.995389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4b896-ed0d-48d7-9989-6468b71ccb5f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:28.995674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4b896-ed0d-48d7-9989-6468b71ccb5f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:28.995947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4b896-ed0d-48d7-9989-6468b71ccb5f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:28.997322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 941/1892] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:18:29.030792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d74b7-ea07-4655-9fde-945a688c4fcc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:29.031070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d74b7-ea07-4655-9fde-945a688c4fcc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:29.031337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d74b7-ea07-4655-9fde-945a688c4fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:29.067987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0d8d74b7-ea07-4655-9fde-945a688c4fcc None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:29.069139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0d8d74b7-ea07-4655-9fde-945a688c4fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:29.081737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f0d5b-0d3d-4bf7-a2f1-e5ac2f2e005c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:29.087083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:29.088222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:29.088902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:29.089492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:29.102540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:29.104711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:29.126612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 952/1893] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:29.202719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 942/1894] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:18:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:29.457946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:29.459597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:29.478038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f44e0-ce5d-4fbc-922d-642c96a55254 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:29.483073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d74b7-ea07-4655-9fde-945a688c4fcc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:29.484506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0d8d74b7-ea07-4655-9fde-945a688c4fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:29.492842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:29.494124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:29.494669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:29.495159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:29.506613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d74b7-ea07-4655-9fde-945a688c4fcc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:29.512723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:29.516837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:29.521368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:29.522033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:29.522309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:29.522576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] PATH_INFO: `/v3/users/9f81a268eebc4ba88cbb75fe94f5ec95` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:29.527290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f81a268eebc4ba88cbb75fe94f5ec95)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:29.529278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:29.537156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 953/1895] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:29.561399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:29.582087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:29.586383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:29.606090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f81a268eebc4ba88cbb75fe94f5ec95'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.634212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f81a268eebc4ba88cbb75fe94f5ec95'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.634593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1da1d-4616-4c30-94f4-6131166e17d5 service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:29.635049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f81a268eebc4ba88cbb75fe94f5ec95'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.635807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f81a268eebc4ba88cbb75fe94f5ec95'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.642851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:29.643690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:29.644127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:29.644561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] PATH_INFO: `/v3/projects/6ea37130fa0f44918cfd4523eb445a36` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:29.646804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0a2877bd-2100-4721-89f8-b3beb3b712d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f81a268eebc4ba88cbb75fe94f5ec95'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.646989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea37130fa0f44918cfd4523eb445a36)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:29.649359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:29.671064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 943/1896] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:29 2022] DELETE /identity/v3/users/9f81a268eebc4ba88cbb75fe94f5ec95 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:29.676901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a73ed-ae90-4873-9576-2ea7053cf45e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:29.678044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a73ed-ae90-4873-9576-2ea7053cf45e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:29.678712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a73ed-ae90-4873-9576-2ea7053cf45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:29.709944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-db8a73ed-ae90-4873-9576-2ea7053cf45e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:29.712545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-db8a73ed-ae90-4873-9576-2ea7053cf45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:29.729555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:29.734434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea37130fa0f44918cfd4523eb445a36', 'request_id': 'req-3122487a-e82c-430b-91c4-295df5e486c2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.746412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea37130fa0f44918cfd4523eb445a36', 'request_id': 'req-3122487a-e82c-430b-91c4-295df5e486c2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.755148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea37130fa0f44918cfd4523eb445a36', 'request_id': 'req-3122487a-e82c-430b-91c4-295df5e486c2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.798515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] The token cache is being invalidate because project 6ea37130fa0f44918cfd4523eb445a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:29.802061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.803258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:29.806431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 954/1897] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:29 2022] DELETE /identity/v3/projects/6ea37130fa0f44918cfd4523eb445a36 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:29.810611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:29.812764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:29.831841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 944/1898] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:29.843286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a73ed-ae90-4873-9576-2ea7053cf45e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:29.845063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-db8a73ed-ae90-4873-9576-2ea7053cf45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:29.956225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122487a-e82c-430b-91c4-295df5e486c2 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:29.965759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:29.966379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:29.966752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:29.967091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] PATH_INFO: `/v3/users/8a62d79ddd9740fdb8fd61f273ecf037` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:29.968795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a62d79ddd9740fdb8fd61f273ecf037)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:29.970914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:29.971172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a73ed-ae90-4873-9576-2ea7053cf45e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:29.980394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:29.980891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:29.981402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:29.981875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:29.995117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:29.995899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:30.009221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 945/1899] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:30.031125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a62d79ddd9740fdb8fd61f273ecf037'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.040219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a62d79ddd9740fdb8fd61f273ecf037'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.041089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a62d79ddd9740fdb8fd61f273ecf037'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.059468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a62d79ddd9740fdb8fd61f273ecf037'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.061021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a62d79ddd9740fdb8fd61f273ecf037'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.081151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 955/1900] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:29 2022] DELETE /identity/v3/users/8a62d79ddd9740fdb8fd61f273ecf037 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:30.333795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:30.335216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:30.349348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae1f4c-5036-48a8-8b20-c6b463558a2a service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:30.394402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:30.418409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:30.419074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:30.419354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:30.419620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] PATH_INFO: `/v3/projects/fef3ab8cfef54a8aa47767dd254a2c0c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:30.420784 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef3ab8cfef54a8aa47767dd254a2c0c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:30.422712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:30.484775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:30.486106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef3ab8cfef54a8aa47767dd254a2c0c', 'request_id': 'req-4e91a617-f7fd-418d-9ca9-74dc39fd896f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.486242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:30.499810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef3ab8cfef54a8aa47767dd254a2c0c', 'request_id': 'req-4e91a617-f7fd-418d-9ca9-74dc39fd896f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.504469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53e941-e80d-4f61-a2bb-1b58c35b06c9 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:30.508588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef3ab8cfef54a8aa47767dd254a2c0c', 'request_id': 'req-4e91a617-f7fd-418d-9ca9-74dc39fd896f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.524552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] The token cache is being invalidate because project fef3ab8cfef54a8aa47767dd254a2c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:30.524894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.525711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.527759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 946/1901] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:30 2022] DELETE /identity/v3/projects/fef3ab8cfef54a8aa47767dd254a2c0c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:30.533609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:30.535163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:30.578558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:30.591089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:30.591348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:30.591444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:30.591677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] PATH_INFO: `/v3/projects/a4b1bf4b145f450484db875c786c1147` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:30.593056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b1bf4b145f450484db875c786c1147)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:30.596055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:30.622572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a617-f7fd-418d-9ca9-74dc39fd896f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:30.627649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:30.628294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:30.628728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:30.630126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] PATH_INFO: `/v3/users/f54904f099d84d7991ba2a74dda9f54c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:30.630987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54904f099d84d7991ba2a74dda9f54c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:30.633106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:30.659996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b1bf4b145f450484db875c786c1147', 'request_id': 'req-afe46a9a-44bf-4669-9972-115bb3e33a72'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.668145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b1bf4b145f450484db875c786c1147', 'request_id': 'req-afe46a9a-44bf-4669-9972-115bb3e33a72'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.677338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b1bf4b145f450484db875c786c1147', 'request_id': 'req-afe46a9a-44bf-4669-9972-115bb3e33a72'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.692826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] The token cache is being invalidate because project a4b1bf4b145f450484db875c786c1147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:30.693408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.694245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-afe46a9a-44bf-4669-9972-115bb3e33a72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.695345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54904f099d84d7991ba2a74dda9f54c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.696196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 956/1902] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:30 2022] DELETE /identity/v3/projects/a4b1bf4b145f450484db875c786c1147 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:30.700954 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86b40b-ccca-4e9f-9bb9-6eea35af2d14 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:30.701309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86b40b-ccca-4e9f-9bb9-6eea35af2d14 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:30.701611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86b40b-ccca-4e9f-9bb9-6eea35af2d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:30.722335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fa86b40b-ccca-4e9f-9bb9-6eea35af2d14 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:30.724097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fa86b40b-ccca-4e9f-9bb9-6eea35af2d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:30.736878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54904f099d84d7991ba2a74dda9f54c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.736878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54904f099d84d7991ba2a74dda9f54c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.736878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54904f099d84d7991ba2a74dda9f54c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.742694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3400752-4990-4b64-9cbf-77bafc2e3a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54904f099d84d7991ba2a74dda9f54c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:30.744277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:30.758263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 947/1903] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:30 2022] DELETE /identity/v3/users/f54904f099d84d7991ba2a74dda9f54c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:30.798004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 957/1904] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:30.807213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8438-cf3a-4f9e-9757-c36eaa2edffb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:30.807503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8438-cf3a-4f9e-9757-c36eaa2edffb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:30.807817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8438-cf3a-4f9e-9757-c36eaa2edffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:30.828410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c65c8438-cf3a-4f9e-9757-c36eaa2edffb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:30.829662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c65c8438-cf3a-4f9e-9757-c36eaa2edffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:30.898826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 948/1905] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:30.903185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86b40b-ccca-4e9f-9bb9-6eea35af2d14 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:30.915329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86b40b-ccca-4e9f-9bb9-6eea35af2d14 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:30.920628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:30.921614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:30.922077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:30.922599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:30.923418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:30.923853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:30.925717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:30.945994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 958/1906] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:30.947099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c8438-cf3a-4f9e-9757-c36eaa2edffb None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:30.956817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c8438-cf3a-4f9e-9757-c36eaa2edffb None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:30.963263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6321a-4bca-46c7-8383-fb0166fa332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:30.964135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6321a-4bca-46c7-8383-fb0166fa332c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:30.964670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6321a-4bca-46c7-8383-fb0166fa332c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:30.965017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6321a-4bca-46c7-8383-fb0166fa332c admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:30.965730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6321a-4bca-46c7-8383-fb0166fa332c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:30.967929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6321a-4bca-46c7-8383-fb0166fa332c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.007521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 949/1907] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:30 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:31.011690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.021842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df31fc33-f0a5-423f-8147-e94bc2492ed0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.087205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23dccd-1df7-431d-96ca-01e2e8c00155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.087836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23dccd-1df7-431d-96ca-01e2e8c00155 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.088270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23dccd-1df7-431d-96ca-01e2e8c00155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.088543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23dccd-1df7-431d-96ca-01e2e8c00155 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.089342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23dccd-1df7-431d-96ca-01e2e8c00155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.091545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23dccd-1df7-431d-96ca-01e2e8c00155 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.120536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:31.131262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6321a-4bca-46c7-8383-fb0166fa332c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.142607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 959/1908] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:31 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:31.142882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6321a-4bca-46c7-8383-fb0166fa332c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.159230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61f98-76f1-4725-970b-5e68b657cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.160117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61f98-76f1-4725-970b-5e68b657cdcf admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.161063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61f98-76f1-4725-970b-5e68b657cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.161345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61f98-76f1-4725-970b-5e68b657cdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.162017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c61f98-76f1-4725-970b-5e68b657cdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.163925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c61f98-76f1-4725-970b-5e68b657cdcf admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.175350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 950/1909] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:18:31.180407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23dccd-1df7-431d-96ca-01e2e8c00155 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.191198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23dccd-1df7-431d-96ca-01e2e8c00155 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.196887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709dff4-885d-4bf3-9702-d4bdf19e9aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.197639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709dff4-885d-4bf3-9702-d4bdf19e9aca admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.197981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709dff4-885d-4bf3-9702-d4bdf19e9aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.198318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709dff4-885d-4bf3-9702-d4bdf19e9aca admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.199041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709dff4-885d-4bf3-9702-d4bdf19e9aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.200767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709dff4-885d-4bf3-9702-d4bdf19e9aca admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.215098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 960/1910] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:31.219510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61f98-76f1-4725-970b-5e68b657cdcf admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.230388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61f98-76f1-4725-970b-5e68b657cdcf admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.235008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.235657 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.235932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.236344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] PATH_INFO: `/v3/projects/85c019992192497d9832594a889da9bf/users/854d5db6c7f64422a946d5284570e1be/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.255069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c019992192497d9832594a889da9bf, user_id=854d5db6c7f64422a946d5284570e1be, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.257378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.276607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 951/1911] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:31 2022] PUT /identity/v3/projects/85c019992192497d9832594a889da9bf/users/854d5db6c7f64422a946d5284570e1be/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:31.278572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb11e1-9f35-46cf-be6b-f76c2f82f488 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.278845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb11e1-9f35-46cf-be6b-f76c2f82f488 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.279135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb11e1-9f35-46cf-be6b-f76c2f82f488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.301963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8feb11e1-9f35-46cf-be6b-f76c2f82f488 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:31.303358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8feb11e1-9f35-46cf-be6b-f76c2f82f488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:31.361725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.364603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:31.373759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 961/1912] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:31.387539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb11e1-9f35-46cf-be6b-f76c2f82f488 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.389724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8feb11e1-9f35-46cf-be6b-f76c2f82f488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:31.469609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b823cae-4cc9-4446-8695-c4d02a3cc413 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.477931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.477931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.477931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.478297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] PATH_INFO: `/v3/projects/17188ef625ce46c5ad9215d991c646e6` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.479326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17188ef625ce46c5ad9215d991c646e6)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.481292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.489947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb11e1-9f35-46cf-be6b-f76c2f82f488 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.494554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4bab3-f6b8-4f56-811f-ab70371bda01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.495309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4bab3-f6b8-4f56-811f-ab70371bda01 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.495691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4bab3-f6b8-4f56-811f-ab70371bda01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.496066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4bab3-f6b8-4f56-811f-ab70371bda01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.507884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4bab3-f6b8-4f56-811f-ab70371bda01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.509573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4bab3-f6b8-4f56-811f-ab70371bda01 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.522536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 962/1913] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:31.530702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:31.545044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17188ef625ce46c5ad9215d991c646e6', 'request_id': 'req-2e44288c-ed88-4215-a4be-9286fc9260e5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:31.557549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17188ef625ce46c5ad9215d991c646e6', 'request_id': 'req-2e44288c-ed88-4215-a4be-9286fc9260e5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:31.567516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17188ef625ce46c5ad9215d991c646e6', 'request_id': 'req-2e44288c-ed88-4215-a4be-9286fc9260e5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:31.579034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] The token cache is being invalidate because project 17188ef625ce46c5ad9215d991c646e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:31.579461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:31.580661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2e44288c-ed88-4215-a4be-9286fc9260e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:31.584202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 952/1914] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:31 2022] DELETE /identity/v3/projects/17188ef625ce46c5ad9215d991c646e6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:31.589059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc6cdd-dd99-40d3-9c84-7fbe070497c0 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.589563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc6cdd-dd99-40d3-9c84-7fbe070497c0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.589662 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc6cdd-dd99-40d3-9c84-7fbe070497c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.611452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e7cc6cdd-dd99-40d3-9c84-7fbe070497c0 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:31.612998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e7cc6cdd-dd99-40d3-9c84-7fbe070497c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:31.707044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 963/1915] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:31.715602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41abf2d-7079-4845-b0bc-22ea79b0839d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.715935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41abf2d-7079-4845-b0bc-22ea79b0839d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.716214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41abf2d-7079-4845-b0bc-22ea79b0839d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.748518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d41abf2d-7079-4845-b0bc-22ea79b0839d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:31.750145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d41abf2d-7079-4845-b0bc-22ea79b0839d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:31.778055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 953/1916] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:31.782147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc6cdd-dd99-40d3-9c84-7fbe070497c0 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.797929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc6cdd-dd99-40d3-9c84-7fbe070497c0 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.810134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.810920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.811384 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.811755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.812586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:31.812995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.816030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.831639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 964/1917] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:31.836009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41abf2d-7079-4845-b0bc-22ea79b0839d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.846849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41abf2d-7079-4845-b0bc-22ea79b0839d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.853020 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba801b-034b-4291-b8b2-432dba1f9349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:31.853887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba801b-034b-4291-b8b2-432dba1f9349 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:31.854157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba801b-034b-4291-b8b2-432dba1f9349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:31.855676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba801b-034b-4291-b8b2-432dba1f9349 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:31.856329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba801b-034b-4291-b8b2-432dba1f9349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:31.858337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba801b-034b-4291-b8b2-432dba1f9349 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:31.907448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 954/1918] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:31 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:31.912107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:31.923385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082361a0-1b42-464e-b9f5-0d8a0fb2e171 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:31.964433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:32.059771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be0c31-e06f-4b7f-bc4b-c8f8b14cbe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.060860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0c31-e06f-4b7f-bc4b-c8f8b14cbe04 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.060860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0c31-e06f-4b7f-bc4b-c8f8b14cbe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.061134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0c31-e06f-4b7f-bc4b-c8f8b14cbe04 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.061880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be0c31-e06f-4b7f-bc4b-c8f8b14cbe04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.064253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be0c31-e06f-4b7f-bc4b-c8f8b14cbe04 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.093642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 965/1919] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:31 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:18:32.094849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba801b-034b-4291-b8b2-432dba1f9349 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:32.107728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba801b-034b-4291-b8b2-432dba1f9349 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:32.113130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d84189-0cf0-4004-a8f2-e4cd7c559f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.114421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d84189-0cf0-4004-a8f2-e4cd7c559f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.114686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d84189-0cf0-4004-a8f2-e4cd7c559f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.114956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d84189-0cf0-4004-a8f2-e4cd7c559f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.115604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d84189-0cf0-4004-a8f2-e4cd7c559f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.117266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d84189-0cf0-4004-a8f2-e4cd7c559f0e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.139314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 955/1920] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:32.144123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be0c31-e06f-4b7f-bc4b-c8f8b14cbe04 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:32.155182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be0c31-e06f-4b7f-bc4b-c8f8b14cbe04 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:32.161150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de51e88-04cf-4c23-9d99-7fbe837f0b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.161880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de51e88-04cf-4c23-9d99-7fbe837f0b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.162164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de51e88-04cf-4c23-9d99-7fbe837f0b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.162470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de51e88-04cf-4c23-9d99-7fbe837f0b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.163333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de51e88-04cf-4c23-9d99-7fbe837f0b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.165254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de51e88-04cf-4c23-9d99-7fbe837f0b49 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.178545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 966/1921] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:32.183103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d84189-0cf0-4004-a8f2-e4cd7c559f0e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:32.194271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d84189-0cf0-4004-a8f2-e4cd7c559f0e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:32.199174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9b404-a72d-467b-864e-3b855ed8be43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.199868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b404-a72d-467b-864e-3b855ed8be43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.200154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b404-a72d-467b-864e-3b855ed8be43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.200429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9b404-a72d-467b-864e-3b855ed8be43 admin admin] PATH_INFO: `/v3/projects/07b1ed9f0f8e43faa63cd25bbea7c57b/users/924249765358424d9c2e3cf8616f6e3e/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.217634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9b404-a72d-467b-864e-3b855ed8be43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b1ed9f0f8e43faa63cd25bbea7c57b, user_id=924249765358424d9c2e3cf8616f6e3e, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.220390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9b404-a72d-467b-864e-3b855ed8be43 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.240056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 956/1922] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:32 2022] PUT /identity/v3/projects/07b1ed9f0f8e43faa63cd25bbea7c57b/users/924249765358424d9c2e3cf8616f6e3e/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:32.249628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c962-a936-4164-8ca0-3f65eec68be6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.249816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c962-a936-4164-8ca0-3f65eec68be6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.250224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163c962-a936-4164-8ca0-3f65eec68be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.283305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-0163c962-a936-4164-8ca0-3f65eec68be6 None None] MFA Rules not processed for user `924249765358424d9c2e3cf8616f6e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:32.287161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0163c962-a936-4164-8ca0-3f65eec68be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:32.377326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 967/1923] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:32.383040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9b404-a72d-467b-864e-3b855ed8be43 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:32.477673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9b404-a72d-467b-864e-3b855ed8be43 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:32.485993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadb078-7ffb-4306-8f74-6cb2f5b03ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.487631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadb078-7ffb-4306-8f74-6cb2f5b03ede admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.487999 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadb078-7ffb-4306-8f74-6cb2f5b03ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.488345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadb078-7ffb-4306-8f74-6cb2f5b03ede admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.489235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadb078-7ffb-4306-8f74-6cb2f5b03ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.494325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadb078-7ffb-4306-8f74-6cb2f5b03ede admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.531202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 957/1924] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:32 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:18:32.534806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163c962-a936-4164-8ca0-3f65eec68be6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:32.558344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163c962-a936-4164-8ca0-3f65eec68be6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:32.649229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195d01f-dcfa-481b-9a11-48d711c8bcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.649863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195d01f-dcfa-481b-9a11-48d711c8bcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.650130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195d01f-dcfa-481b-9a11-48d711c8bcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.650400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195d01f-dcfa-481b-9a11-48d711c8bcd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.651078 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195d01f-dcfa-481b-9a11-48d711c8bcd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.653050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195d01f-dcfa-481b-9a11-48d711c8bcd0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.684998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 968/1925] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:32 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:32.687670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadb078-7ffb-4306-8f74-6cb2f5b03ede admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:32.729206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadb078-7ffb-4306-8f74-6cb2f5b03ede admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:32.745049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf866-30a0-477e-9598-f435f31678c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.747774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf866-30a0-477e-9598-f435f31678c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.748075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf866-30a0-477e-9598-f435f31678c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.748357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdf866-30a0-477e-9598-f435f31678c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.755494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdf866-30a0-477e-9598-f435f31678c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.758529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdf866-30a0-477e-9598-f435f31678c6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.787363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 958/1926] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:32.793235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195d01f-dcfa-481b-9a11-48d711c8bcd0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:32.827109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195d01f-dcfa-481b-9a11-48d711c8bcd0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:32.844903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad2532-44d2-4bc6-b20f-74ec99416f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.844903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad2532-44d2-4bc6-b20f-74ec99416f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.844903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad2532-44d2-4bc6-b20f-74ec99416f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.844903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad2532-44d2-4bc6-b20f-74ec99416f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.845443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad2532-44d2-4bc6-b20f-74ec99416f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.845443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad2532-44d2-4bc6-b20f-74ec99416f8b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.868051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 969/1927] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:32.872282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf866-30a0-477e-9598-f435f31678c6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:32.886975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdf866-30a0-477e-9598-f435f31678c6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:32.892264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0213f-0070-4e42-b10b-c5881a041ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:32.893215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0213f-0070-4e42-b10b-c5881a041ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.893973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0213f-0070-4e42-b10b-c5881a041ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.894510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0213f-0070-4e42-b10b-c5881a041ec5 admin admin] PATH_INFO: `/v3/projects/2a8efa1af4de4dceb0c0cc3a9ca677bf/users/9588d17614fa433698b282b72c337504/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.915330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0213f-0070-4e42-b10b-c5881a041ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8efa1af4de4dceb0c0cc3a9ca677bf, user_id=9588d17614fa433698b282b72c337504, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:32.920448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0213f-0070-4e42-b10b-c5881a041ec5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:32.948683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 959/1928] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:32 2022] PUT /identity/v3/projects/2a8efa1af4de4dceb0c0cc3a9ca677bf/users/9588d17614fa433698b282b72c337504/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:32.964707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ef2da-bc91-43cd-b2a8-67d7beef716a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:32.965053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ef2da-bc91-43cd-b2a8-67d7beef716a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:32.965288 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ef2da-bc91-43cd-b2a8-67d7beef716a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:32.992820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:33.015127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f67ef2da-bc91-43cd-b2a8-67d7beef716a None None] MFA Rules not processed for user `9588d17614fa433698b282b72c337504`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:33.016434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f67ef2da-bc91-43cd-b2a8-67d7beef716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:33.199014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 970/1929] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:33.201441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0213f-0070-4e42-b10b-c5881a041ec5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:33.215321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:33.327790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0213f-0070-4e42-b10b-c5881a041ec5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:33.341200 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89816e-d0b8-485a-a108-d693c0daf458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:33.342226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89816e-d0b8-485a-a108-d693c0daf458 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:33.342679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89816e-d0b8-485a-a108-d693c0daf458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:33.343084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89816e-d0b8-485a-a108-d693c0daf458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:33.343868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89816e-d0b8-485a-a108-d693c0daf458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:33.346974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89816e-d0b8-485a-a108-d693c0daf458 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:33.400421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 960/1930] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:33 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:33.401740 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ef2da-bc91-43cd-b2a8-67d7beef716a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:33.420141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ef2da-bc91-43cd-b2a8-67d7beef716a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:33.569865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde5011-f367-4bf1-b4a1-45cdd12dd148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:33.572924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5011-f367-4bf1-b4a1-45cdd12dd148 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:33.573242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5011-f367-4bf1-b4a1-45cdd12dd148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:33.573648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5011-f367-4bf1-b4a1-45cdd12dd148 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:33.574330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde5011-f367-4bf1-b4a1-45cdd12dd148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:33.578180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde5011-f367-4bf1-b4a1-45cdd12dd148 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:33.616272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 971/1931] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:33 2022] POST /identity/v3/users => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:33.618003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89816e-d0b8-485a-a108-d693c0daf458 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:33.632344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89816e-d0b8-485a-a108-d693c0daf458 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:33.640051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806ecf6-8030-4084-831d-c12a945100ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:33.641084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ecf6-8030-4084-831d-c12a945100ca admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:33.641462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ecf6-8030-4084-831d-c12a945100ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:33.642364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ecf6-8030-4084-831d-c12a945100ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:33.642539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806ecf6-8030-4084-831d-c12a945100ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:33.644341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806ecf6-8030-4084-831d-c12a945100ca admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:33.659419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 961/1932] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:33.665378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde5011-f367-4bf1-b4a1-45cdd12dd148 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:33.678933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde5011-f367-4bf1-b4a1-45cdd12dd148 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:33.688222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be42e2e-1f93-4bd1-9408-c00be18056bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:33.695153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42e2e-1f93-4bd1-9408-c00be18056bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:33.695153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42e2e-1f93-4bd1-9408-c00be18056bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:33.695153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42e2e-1f93-4bd1-9408-c00be18056bd admin admin] PATH_INFO: `/v3/projects/b66f2f5ebdef4b55a01029715868056c/users/ba39324b6ad741d7965f3d1ce0bb6ea8/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:33.725730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be42e2e-1f93-4bd1-9408-c00be18056bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66f2f5ebdef4b55a01029715868056c, user_id=ba39324b6ad741d7965f3d1ce0bb6ea8, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:33.725730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be42e2e-1f93-4bd1-9408-c00be18056bd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:33.744599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 972/1933] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:33 2022] PUT /identity/v3/projects/b66f2f5ebdef4b55a01029715868056c/users/ba39324b6ad741d7965f3d1ce0bb6ea8/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:33.750232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806ecf6-8030-4084-831d-c12a945100ca admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:33.790709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806ecf6-8030-4084-831d-c12a945100ca admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:33.893613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e157f75-6a8a-4eb8-a0e0-ff983129967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:33.894347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e157f75-6a8a-4eb8-a0e0-ff983129967a admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:33.894654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e157f75-6a8a-4eb8-a0e0-ff983129967a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:33.895603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e157f75-6a8a-4eb8-a0e0-ff983129967a admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:33.896275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e157f75-6a8a-4eb8-a0e0-ff983129967a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:33.898267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e157f75-6a8a-4eb8-a0e0-ff983129967a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:33.919782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 962/1934] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:33.921014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be42e2e-1f93-4bd1-9408-c00be18056bd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:33.945318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be42e2e-1f93-4bd1-9408-c00be18056bd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:33.950652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:33.951265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:33.951797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:33.952049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] PATH_INFO: `/v3/projects/b66f2f5ebdef4b55a01029715868056c/users/ba39324b6ad741d7965f3d1ce0bb6ea8/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:33.954533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66f2f5ebdef4b55a01029715868056c, user_id=ba39324b6ad741d7965f3d1ce0bb6ea8, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:33.958282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:33.970564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 973/1935] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:33 2022] PUT /identity/v3/projects/b66f2f5ebdef4b55a01029715868056c/users/ba39324b6ad741d7965f3d1ce0bb6ea8/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:33.982391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17737-cfcf-4468-ba08-9da6b1354b87 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:33.982752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17737-cfcf-4468-ba08-9da6b1354b87 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:33.983038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb17737-cfcf-4468-ba08-9da6b1354b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:34.017364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4bb17737-cfcf-4468-ba08-9da6b1354b87 None None] MFA Rules not processed for user `ba39324b6ad741d7965f3d1ce0bb6ea8`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:34.018521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4bb17737-cfcf-4468-ba08-9da6b1354b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:34.149963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 963/1936] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:34.162532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:34.164511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:34.242779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:34.249455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36188a51-8f4a-41e1-af16-8c96329153e9 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:34.255304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:34.256535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:34.256755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:34.257133 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:34.328439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:34.330049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:34.341595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 974/1937] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:37.244337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c45ef-531b-4c52-a4ce-80b806927abd None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:37.245169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c45ef-531b-4c52-a4ce-80b806927abd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:37.245614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c45ef-531b-4c52-a4ce-80b806927abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:37.259050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:37.309530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2c3c45ef-531b-4c52-a4ce-80b806927abd None None] MFA Rules not processed for user `854d5db6c7f64422a946d5284570e1be`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:37.310837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2c3c45ef-531b-4c52-a4ce-80b806927abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:37.471266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:37.471804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 964/1938] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:37.473403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:37.560270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702d85a-36ec-4ebe-9597-2d9895b9324c service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:37.566388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c55ea-23c6-4eb0-9d41-24e55984ba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:37.567798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c55ea-23c6-4eb0-9d41-24e55984ba1c admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:37.568312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c55ea-23c6-4eb0-9d41-24e55984ba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:37.568630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c55ea-23c6-4eb0-9d41-24e55984ba1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:37.569395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c55ea-23c6-4eb0-9d41-24e55984ba1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:37.571529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c55ea-23c6-4eb0-9d41-24e55984ba1c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:37.617800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 975/1939] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:37 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:37.619028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c45ef-531b-4c52-a4ce-80b806927abd None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:37.641905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c45ef-531b-4c52-a4ce-80b806927abd None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:37.777531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6408625-a842-4473-9374-5c9cd94e269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:37.778160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6408625-a842-4473-9374-5c9cd94e269e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:37.778440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6408625-a842-4473-9374-5c9cd94e269e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:37.778714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6408625-a842-4473-9374-5c9cd94e269e admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:37.779358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6408625-a842-4473-9374-5c9cd94e269e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:37.781932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6408625-a842-4473-9374-5c9cd94e269e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:37.833869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c55ea-23c6-4eb0-9d41-24e55984ba1c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:37.834774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 965/1940] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:37 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:37.846462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c55ea-23c6-4eb0-9d41-24e55984ba1c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:37.855918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe740829-e445-434a-9dc5-d37158587abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:37.856671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe740829-e445-434a-9dc5-d37158587abd admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:37.857741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe740829-e445-434a-9dc5-d37158587abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:37.858010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe740829-e445-434a-9dc5-d37158587abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:37.858854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe740829-e445-434a-9dc5-d37158587abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:37.864842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe740829-e445-434a-9dc5-d37158587abd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:37.874911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 976/1941] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:37.880792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6408625-a842-4473-9374-5c9cd94e269e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:37.892748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6408625-a842-4473-9374-5c9cd94e269e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:37.899249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b617509-626b-406a-a713-dac3ea3f241b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:37.900181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b617509-626b-406a-a713-dac3ea3f241b admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:37.900550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b617509-626b-406a-a713-dac3ea3f241b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:37.901760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b617509-626b-406a-a713-dac3ea3f241b admin admin] PATH_INFO: `/v3/projects/cea2433debb24b698dd2836acd25996e/users/080a65c2609a4d2bad406378af30bfa8/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:37.924853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b617509-626b-406a-a713-dac3ea3f241b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea2433debb24b698dd2836acd25996e, user_id=080a65c2609a4d2bad406378af30bfa8, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:37.928038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b617509-626b-406a-a713-dac3ea3f241b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:37.985264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 966/1942] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:37 2022] PUT /identity/v3/projects/cea2433debb24b698dd2836acd25996e/users/080a65c2609a4d2bad406378af30bfa8/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:37.987324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe740829-e445-434a-9dc5-d37158587abd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:38.005258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe740829-e445-434a-9dc5-d37158587abd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:38.159196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:38.159871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:38.160516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:38.160928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:38.161670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:38.164295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:38.184733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b617509-626b-406a-a713-dac3ea3f241b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:38.199179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 977/1943] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:38.216349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b617509-626b-406a-a713-dac3ea3f241b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:38.230841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5169d8a-ff53-4736-9e12-23abec8147f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:38.231533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5169d8a-ff53-4736-9e12-23abec8147f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:38.231885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5169d8a-ff53-4736-9e12-23abec8147f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:38.232072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5169d8a-ff53-4736-9e12-23abec8147f6 admin admin] PATH_INFO: `/v3/projects/cea2433debb24b698dd2836acd25996e/users/080a65c2609a4d2bad406378af30bfa8/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:38.242878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5169d8a-ff53-4736-9e12-23abec8147f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea2433debb24b698dd2836acd25996e, user_id=080a65c2609a4d2bad406378af30bfa8, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:38.245692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5169d8a-ff53-4736-9e12-23abec8147f6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:38.269866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 967/1944] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:38 2022] PUT /identity/v3/projects/cea2433debb24b698dd2836acd25996e/users/080a65c2609a4d2bad406378af30bfa8/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:38.340330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:38.344472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:38.375495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:38.489974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca7b6-9b21-4a1f-b5be-6262b3613ba7 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:38.568121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804240a-7ee2-4108-89e7-3a80149cb62a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:38.568380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804240a-7ee2-4108-89e7-3a80149cb62a service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:38.568654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804240a-7ee2-4108-89e7-3a80149cb62a service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:38.568928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804240a-7ee2-4108-89e7-3a80149cb62a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:38.659072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804240a-7ee2-4108-89e7-3a80149cb62a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:38.663447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804240a-7ee2-4108-89e7-3a80149cb62a service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:38.705107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 978/1945] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:39.616430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5169d8a-ff53-4736-9e12-23abec8147f6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:39.631920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5169d8a-ff53-4736-9e12-23abec8147f6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:39.638451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245cf2-a203-4ea4-a263-452a09df3327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:39.639248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245cf2-a203-4ea4-a263-452a09df3327 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:39.640395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245cf2-a203-4ea4-a263-452a09df3327 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:39.640857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245cf2-a203-4ea4-a263-452a09df3327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:39.720885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:39.751209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245cf2-a203-4ea4-a263-452a09df3327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:39.752900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a245cf2-a203-4ea4-a263-452a09df3327 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:39.767464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 968/1946] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:44.659574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804240a-7ee2-4108-89e7-3a80149cb62a service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:44.797859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804240a-7ee2-4108-89e7-3a80149cb62a service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:44.811936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:44.812918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:44.813263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:44.814171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] PATH_INFO: `/v3/users/924249765358424d9c2e3cf8616f6e3e` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:44.818726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=924249765358424d9c2e3cf8616f6e3e)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:44.821145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:44.871293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:44.906453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '924249765358424d9c2e3cf8616f6e3e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:44.922493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '924249765358424d9c2e3cf8616f6e3e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:44.923343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '924249765358424d9c2e3cf8616f6e3e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:44.949290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '924249765358424d9c2e3cf8616f6e3e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:44.951731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '924249765358424d9c2e3cf8616f6e3e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:44.971850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 979/1947] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:44 2022] DELETE /identity/v3/users/924249765358424d9c2e3cf8616f6e3e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:44.979588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893eaaf-ad65-4082-b5d7-22768035cdc7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:44.980705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893eaaf-ad65-4082-b5d7-22768035cdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:44.981307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893eaaf-ad65-4082-b5d7-22768035cdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:45.007247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8893eaaf-ad65-4082-b5d7-22768035cdc7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:45.009121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8893eaaf-ad65-4082-b5d7-22768035cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:45.110271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 969/1948] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:45.148403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:45.154262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:45.259221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e95c6-ce0f-4f32-9da4-6260dd6f4dc6 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:45.263817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc165c94-28fb-4cb7-ab4a-471b6b4b2d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:45.264929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165c94-28fb-4cb7-ab4a-471b6b4b2d5f service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:45.265373 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165c94-28fb-4cb7-ab4a-471b6b4b2d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:45.265789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165c94-28fb-4cb7-ab4a-471b6b4b2d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:45.292227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc165c94-28fb-4cb7-ab4a-471b6b4b2d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:45.294446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc165c94-28fb-4cb7-ab4a-471b6b4b2d5f service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:45.320987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 980/1949] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:46.783706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893eaaf-ad65-4082-b5d7-22768035cdc7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:46.787270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8893eaaf-ad65-4082-b5d7-22768035cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:46.802535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893eaaf-ad65-4082-b5d7-22768035cdc7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:46.807338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:46.808211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:46.808559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:46.808930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] PATH_INFO: `/v3/projects/07b1ed9f0f8e43faa63cd25bbea7c57b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:46.810161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b1ed9f0f8e43faa63cd25bbea7c57b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:46.812415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:46.915510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b1ed9f0f8e43faa63cd25bbea7c57b', 'request_id': 'req-085bbf9b-c245-40ee-8fbf-ffa05847c61c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:46.924760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b1ed9f0f8e43faa63cd25bbea7c57b', 'request_id': 'req-085bbf9b-c245-40ee-8fbf-ffa05847c61c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:46.934867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b1ed9f0f8e43faa63cd25bbea7c57b', 'request_id': 'req-085bbf9b-c245-40ee-8fbf-ffa05847c61c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:46.947312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] The token cache is being invalidate because project 07b1ed9f0f8e43faa63cd25bbea7c57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:46.947603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:46.948464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-085bbf9b-c245-40ee-8fbf-ffa05847c61c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:46.950382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 970/1950] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:46 2022] DELETE /identity/v3/projects/07b1ed9f0f8e43faa63cd25bbea7c57b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:46.954510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc165c94-28fb-4cb7-ab4a-471b6b4b2d5f service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:46.966510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc165c94-28fb-4cb7-ab4a-471b6b4b2d5f service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:47.060887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:47.061784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:47.062167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:47.062677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] PATH_INFO: `/v3/users/9588d17614fa433698b282b72c337504` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:47.064131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9588d17614fa433698b282b72c337504)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:47.067402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:47.164737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9588d17614fa433698b282b72c337504'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:47.166446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:47.176881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9588d17614fa433698b282b72c337504'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:47.177491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9588d17614fa433698b282b72c337504'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:47.195191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9588d17614fa433698b282b72c337504'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:47.196115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9588d17614fa433698b282b72c337504'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:47.215596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 981/1951] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:46 2022] DELETE /identity/v3/users/9588d17614fa433698b282b72c337504 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:47.873321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8cbe0-c229-479d-99a1-df97d24e4c5c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:47.873769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8cbe0-c229-479d-99a1-df97d24e4c5c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:47.873936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8cbe0-c229-479d-99a1-df97d24e4c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:47.945651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-aed8cbe0-c229-479d-99a1-df97d24e4c5c None None] MFA Rules not processed for user `080a65c2609a4d2bad406378af30bfa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:47.946925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aed8cbe0-c229-479d-99a1-df97d24e4c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:48.062817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:48.112266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 971/1952] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:48.127254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:48.129791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:48.160157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8cbe0-c229-479d-99a1-df97d24e4c5c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:48.162486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aed8cbe0-c229-479d-99a1-df97d24e4c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:48.325208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c73764-dc75-47e0-a64f-0d04883f04d8 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:48.334499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:48.335301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:48.335687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:48.336061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:48.338154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:48.357129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8cbe0-c229-479d-99a1-df97d24e4c5c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:48.367542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:48.368201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:48.368484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:48.368780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] PATH_INFO: `/v3/projects/2a8efa1af4de4dceb0c0cc3a9ca677bf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:48.372291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8efa1af4de4dceb0c0cc3a9ca677bf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:48.374383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:48.414922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:48.416984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:48.430875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 982/1953] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:48.497559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8efa1af4de4dceb0c0cc3a9ca677bf', 'request_id': 'req-77944ac9-2db0-4607-bb6e-5b44fab9318f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.507891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8efa1af4de4dceb0c0cc3a9ca677bf', 'request_id': 'req-77944ac9-2db0-4607-bb6e-5b44fab9318f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.520376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8efa1af4de4dceb0c0cc3a9ca677bf', 'request_id': 'req-77944ac9-2db0-4607-bb6e-5b44fab9318f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.554275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] The token cache is being invalidate because project 2a8efa1af4de4dceb0c0cc3a9ca677bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:48.554572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.555239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.557028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 972/1954] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:48 2022] DELETE /identity/v3/projects/2a8efa1af4de4dceb0c0cc3a9ca677bf => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:48.568340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:48.580744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207deb-339d-41a0-bf7b-0ad2acdc9fdb service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:48.659623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:48.661572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:48.662048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:48.662504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] PATH_INFO: `/v3/users/ba39324b6ad741d7965f3d1ce0bb6ea8` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:48.664407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba39324b6ad741d7965f3d1ce0bb6ea8)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:48.668199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:48.672100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:48.758994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba39324b6ad741d7965f3d1ce0bb6ea8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.773459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba39324b6ad741d7965f3d1ce0bb6ea8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.774532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba39324b6ad741d7965f3d1ce0bb6ea8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.794561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba39324b6ad741d7965f3d1ce0bb6ea8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.795304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-12d50131-55f5-4121-83fa-ff3298c66ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba39324b6ad741d7965f3d1ce0bb6ea8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:48.820324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 983/1955] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:48 2022] DELETE /identity/v3/users/ba39324b6ad741d7965f3d1ce0bb6ea8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:49.393996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:49.396162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:49.419135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77944ac9-2db0-4607-bb6e-5b44fab9318f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:49.536613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:49.537526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:49.537811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:49.538140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] PATH_INFO: `/v3/projects/b66f2f5ebdef4b55a01029715868056c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:49.539667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66f2f5ebdef4b55a01029715868056c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:49.543027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:49.585723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:49.632171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66f2f5ebdef4b55a01029715868056c', 'request_id': 'req-c36895c7-cbe1-4fdc-86e4-410f12e15570'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:49.649953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66f2f5ebdef4b55a01029715868056c', 'request_id': 'req-c36895c7-cbe1-4fdc-86e4-410f12e15570'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:49.661142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66f2f5ebdef4b55a01029715868056c', 'request_id': 'req-c36895c7-cbe1-4fdc-86e4-410f12e15570'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:49.675263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] The token cache is being invalidate because project b66f2f5ebdef4b55a01029715868056c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:49.676385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:49.677330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c36895c7-cbe1-4fdc-86e4-410f12e15570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:49.683705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 973/1956] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:49 2022] DELETE /identity/v3/projects/b66f2f5ebdef4b55a01029715868056c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:49.685543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993c3e4-97b3-4783-bda0-6c2b7cb8bf29 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:49.686024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993c3e4-97b3-4783-bda0-6c2b7cb8bf29 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:49.686136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993c3e4-97b3-4783-bda0-6c2b7cb8bf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:49.729733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-3993c3e4-97b3-4783-bda0-6c2b7cb8bf29 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:49.731991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3993c3e4-97b3-4783-bda0-6c2b7cb8bf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:49.845543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 984/1957] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:49.848389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70915755-d6b5-4daa-ae19-1b6bd53a6256 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:49.848389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70915755-d6b5-4daa-ae19-1b6bd53a6256 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:49.848389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70915755-d6b5-4daa-ae19-1b6bd53a6256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:49.875478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-70915755-d6b5-4daa-ae19-1b6bd53a6256 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:49.877128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-70915755-d6b5-4daa-ae19-1b6bd53a6256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:49.890429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 974/1958] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:49.891759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993c3e4-97b3-4783-bda0-6c2b7cb8bf29 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:49.904308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993c3e4-97b3-4783-bda0-6c2b7cb8bf29 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:49.912747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:49.913517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:49.914165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:49.914532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:49.915365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:49.915761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:49.918181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:49.933769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 985/1959] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:49.938543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70915755-d6b5-4daa-ae19-1b6bd53a6256 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:49.949974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70915755-d6b5-4daa-ae19-1b6bd53a6256 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:49.956180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd9608-a5aa-4544-a5ec-5f4134b3027d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:49.957561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd9608-a5aa-4544-a5ec-5f4134b3027d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:49.957982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd9608-a5aa-4544-a5ec-5f4134b3027d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:49.958347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd9608-a5aa-4544-a5ec-5f4134b3027d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:49.959264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd9608-a5aa-4544-a5ec-5f4134b3027d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:49.961423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd9608-a5aa-4544-a5ec-5f4134b3027d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:50.000695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 975/1960] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:49 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:50.005065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:50.021297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c30fa-c484-4b56-b31b-b6b912fb62b7 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:50.178860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f21f7c-ebca-4bd5-b042-59cbda576db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:50.179491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21f7c-ebca-4bd5-b042-59cbda576db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:50.180039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21f7c-ebca-4bd5-b042-59cbda576db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:50.180158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21f7c-ebca-4bd5-b042-59cbda576db2 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:50.180967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f21f7c-ebca-4bd5-b042-59cbda576db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:50.183509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f21f7c-ebca-4bd5-b042-59cbda576db2 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:50.208135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:50.223474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 986/1961] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:50 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:50.224830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd9608-a5aa-4544-a5ec-5f4134b3027d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:50.267823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd9608-a5aa-4544-a5ec-5f4134b3027d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:50.276472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f93b75-27cc-499f-9b2a-37a5cb2cfcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:50.277507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f93b75-27cc-499f-9b2a-37a5cb2cfcb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:50.277860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f93b75-27cc-499f-9b2a-37a5cb2cfcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:50.278268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f93b75-27cc-499f-9b2a-37a5cb2cfcb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:50.278985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f93b75-27cc-499f-9b2a-37a5cb2cfcb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:50.280917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f93b75-27cc-499f-9b2a-37a5cb2cfcb3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:50.309759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 976/1962] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:50.314258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f21f7c-ebca-4bd5-b042-59cbda576db2 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:50.338745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f21f7c-ebca-4bd5-b042-59cbda576db2 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:50.355614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0a842-09a8-492e-91dd-bde8e4e32215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:50.357005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a842-09a8-492e-91dd-bde8e4e32215 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:50.357472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a842-09a8-492e-91dd-bde8e4e32215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:50.357898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0a842-09a8-492e-91dd-bde8e4e32215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:50.361711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0a842-09a8-492e-91dd-bde8e4e32215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:50.364445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0a842-09a8-492e-91dd-bde8e4e32215 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:50.389338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 987/1963] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:50.398395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f93b75-27cc-499f-9b2a-37a5cb2cfcb3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:50.409543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f93b75-27cc-499f-9b2a-37a5cb2cfcb3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:50.417502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:50.418556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:50.418942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:50.419453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] PATH_INFO: `/v3/projects/47b4c76757374438b8d4b0b7148be568/users/b7c7e84a2809475cb7fbfaf905c2ad5c/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:50.482278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b4c76757374438b8d4b0b7148be568, user_id=b7c7e84a2809475cb7fbfaf905c2ad5c, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:50.485013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:50.517551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 977/1964] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:50 2022] PUT /identity/v3/projects/47b4c76757374438b8d4b0b7148be568/users/b7c7e84a2809475cb7fbfaf905c2ad5c/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:50.519410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27570d-9510-432d-ba16-a5e7150a5d6e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:50.519946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27570d-9510-432d-ba16-a5e7150a5d6e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:50.520727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27570d-9510-432d-ba16-a5e7150a5d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:50.582183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-cb27570d-9510-432d-ba16-a5e7150a5d6e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:50.585070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cb27570d-9510-432d-ba16-a5e7150a5d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:50.690367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 988/1965] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:50.710376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:50.712263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:50.760481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc0d90-1a0f-465c-8ae9-3c6b976cc4a3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:50.760927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc0d90-1a0f-465c-8ae9-3c6b976cc4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:50.761299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc0d90-1a0f-465c-8ae9-3c6b976cc4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:50.795442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e6fc0d90-1a0f-465c-8ae9-3c6b976cc4a3 None None] MFA Rules not processed for user `854d5db6c7f64422a946d5284570e1be`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:50.796926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e6fc0d90-1a0f-465c-8ae9-3c6b976cc4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:50.905275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 989/1966] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:50.910477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035af510-6693-4568-aee7-70b45a8a89fe admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:50.922470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:50.923947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:50.924242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:50.927386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:50.954714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:50.956611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:50.985791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 978/1967] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:52.562058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8704-8b0f-4774-a222-a992f630f703 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:52.562563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8704-8b0f-4774-a222-a992f630f703 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:52.562688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8704-8b0f-4774-a222-a992f630f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:52.596067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b61d8704-8b0f-4774-a222-a992f630f703 None None] MFA Rules not processed for user `b7c7e84a2809475cb7fbfaf905c2ad5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:52.598063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b61d8704-8b0f-4774-a222-a992f630f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:52.693956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 990/1968] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:52.705594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:52.707936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:52.791874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:52.795348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58263924-2b76-4572-95de-a2c0dc96d25a service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:52.804558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5045a3-7ca3-4c04-a7d7-c8c7fc6e0df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:52.806428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5045a3-7ca3-4c04-a7d7-c8c7fc6e0df8 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:52.807549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5045a3-7ca3-4c04-a7d7-c8c7fc6e0df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:52.808261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5045a3-7ca3-4c04-a7d7-c8c7fc6e0df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:52.827234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5045a3-7ca3-4c04-a7d7-c8c7fc6e0df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:52.829422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5045a3-7ca3-4c04-a7d7-c8c7fc6e0df8 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:52.854677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 979/1969] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:53.420919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d8704-8b0f-4774-a222-a992f630f703 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:53.422451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b61d8704-8b0f-4774-a222-a992f630f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:53.437923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d8704-8b0f-4774-a222-a992f630f703 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:53.442862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:53.443501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:53.443779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:53.444049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] PATH_INFO: `/v3/users/3f691a508e934f92b748cc96022ac195` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:53.445183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f691a508e934f92b748cc96022ac195)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:53.447338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:53.468160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:53.504706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f691a508e934f92b748cc96022ac195'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:53.512010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f691a508e934f92b748cc96022ac195'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:53.512722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f691a508e934f92b748cc96022ac195'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:53.526021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f691a508e934f92b748cc96022ac195'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:53.526717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f691a508e934f92b748cc96022ac195'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:53.549720 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 991/1970] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:53 2022] DELETE /identity/v3/users/3f691a508e934f92b748cc96022ac195 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:53.552623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa79607-dcd9-4ed7-8744-80a133d7aec3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:53.552919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa79607-dcd9-4ed7-8744-80a133d7aec3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:53.553201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa79607-dcd9-4ed7-8744-80a133d7aec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:53.575687 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-caa79607-dcd9-4ed7-8744-80a133d7aec3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:53.576957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-caa79607-dcd9-4ed7-8744-80a133d7aec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:53.662353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 980/1971] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:53.672285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:53.674871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:53.745457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65725d9a-f9bf-4f06-aa7c-485acbd09d6c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:53.749078 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85765e11-822e-4c0b-9336-d4ecc4a1c085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:53.749679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85765e11-822e-4c0b-9336-d4ecc4a1c085 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:53.749956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85765e11-822e-4c0b-9336-d4ecc4a1c085 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:53.750227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85765e11-822e-4c0b-9336-d4ecc4a1c085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:53.760162 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85765e11-822e-4c0b-9336-d4ecc4a1c085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:53.761783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85765e11-822e-4c0b-9336-d4ecc4a1c085 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:53.771395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 992/1972] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:18:54.230170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa79607-dcd9-4ed7-8744-80a133d7aec3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:54.233098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-caa79607-dcd9-4ed7-8744-80a133d7aec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:54.250583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa79607-dcd9-4ed7-8744-80a133d7aec3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:54.258783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:54.259986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:54.260531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:54.261065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] PATH_INFO: `/v3/projects/2fd5594633b7440bb8e0da744194d323` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:54.262503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd5594633b7440bb8e0da744194d323)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:54.264886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:54.333112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469bf30-1855-4a71-9d36-48d9340d4def None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:54.333412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469bf30-1855-4a71-9d36-48d9340d4def None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:54.333666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469bf30-1855-4a71-9d36-48d9340d4def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:54.341594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd5594633b7440bb8e0da744194d323', 'request_id': 'req-ef991546-ddf3-4569-9f32-db886d877fbf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.351162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd5594633b7440bb8e0da744194d323', 'request_id': 'req-ef991546-ddf3-4569-9f32-db886d877fbf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.355590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-5469bf30-1855-4a71-9d36-48d9340d4def None None] MFA Rules not processed for user `b7c7e84a2809475cb7fbfaf905c2ad5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:54.356872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5469bf30-1855-4a71-9d36-48d9340d4def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:54.358406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd5594633b7440bb8e0da744194d323', 'request_id': 'req-ef991546-ddf3-4569-9f32-db886d877fbf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.374703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] The token cache is being invalidate because project 2fd5594633b7440bb8e0da744194d323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:54.375112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.375941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.377737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 981/1973] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:54 2022] DELETE /identity/v3/projects/2fd5594633b7440bb8e0da744194d323 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:54.383083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:54.384883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:54.426721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 993/1974] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:54.473954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef991546-ddf3-4569-9f32-db886d877fbf admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:54.479140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:54.479790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:54.480080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:54.480356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] PATH_INFO: `/v3/users/44a1d4294a4a48efb1b3f97db9810056` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:54.481566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a1d4294a4a48efb1b3f97db9810056)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:54.483511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:54.513531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:54.565192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a1d4294a4a48efb1b3f97db9810056'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.607689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a1d4294a4a48efb1b3f97db9810056'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.608599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a1d4294a4a48efb1b3f97db9810056'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.609327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a1d4294a4a48efb1b3f97db9810056'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.619443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6357b560-44ae-4a65-92df-edcd67f49acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a1d4294a4a48efb1b3f97db9810056'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:54.642485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 982/1975] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:18:54 2022] DELETE /identity/v3/users/44a1d4294a4a48efb1b3f97db9810056 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:55.084447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469bf30-1855-4a71-9d36-48d9340d4def None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:55.097055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469bf30-1855-4a71-9d36-48d9340d4def None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:55.205036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:55.242369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:55.243177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.243918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.243918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] PATH_INFO: `/v3/projects/883f386d473e45faac379bc2482478a9` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.244916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=883f386d473e45faac379bc2482478a9)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:55.246840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:55.313194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883f386d473e45faac379bc2482478a9', 'request_id': 'req-9baa0109-206d-433f-8678-c961a88e5c64'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:55.323424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883f386d473e45faac379bc2482478a9', 'request_id': 'req-9baa0109-206d-433f-8678-c961a88e5c64'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:55.330627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883f386d473e45faac379bc2482478a9', 'request_id': 'req-9baa0109-206d-433f-8678-c961a88e5c64'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:55.349863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] The token cache is being invalidate because project 883f386d473e45faac379bc2482478a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:18:55.350160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:55.350820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9baa0109-206d-433f-8678-c961a88e5c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:18:55.352416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 994/1976] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:18:55 2022] DELETE /identity/v3/projects/883f386d473e45faac379bc2482478a9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:55.357297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31caea9f-3932-4e24-a62c-3a6b15827a49 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.358332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31caea9f-3932-4e24-a62c-3a6b15827a49 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.358816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31caea9f-3932-4e24-a62c-3a6b15827a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.386195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-31caea9f-3932-4e24-a62c-3a6b15827a49 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:55.387604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-31caea9f-3932-4e24-a62c-3a6b15827a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:55.493051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 983/1977] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:18:55.501280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9533ee-a88d-4bc6-b0e8-47b2391f6cec None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.502526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9533ee-a88d-4bc6-b0e8-47b2391f6cec None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.502526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9533ee-a88d-4bc6-b0e8-47b2391f6cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.534543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-9d9533ee-a88d-4bc6-b0e8-47b2391f6cec None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:55.536072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9d9533ee-a88d-4bc6-b0e8-47b2391f6cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:55.549016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 995/1978] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:55.550681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31caea9f-3932-4e24-a62c-3a6b15827a49 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:55.561708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31caea9f-3932-4e24-a62c-3a6b15827a49 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:55.567436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:55.568072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.568424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.568794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.569518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:18:55.569910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:55.571615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:55.585504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 984/1979] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:18:55.591740 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9533ee-a88d-4bc6-b0e8-47b2391f6cec None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:55.606723 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9533ee-a88d-4bc6-b0e8-47b2391f6cec None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:55.615650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbeff3b-1d16-4b20-a93b-b5158d9421fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:55.616626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeff3b-1d16-4b20-a93b-b5158d9421fb admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.616979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeff3b-1d16-4b20-a93b-b5158d9421fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.617319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeff3b-1d16-4b20-a93b-b5158d9421fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.618279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbeff3b-1d16-4b20-a93b-b5158d9421fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:55.621212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbeff3b-1d16-4b20-a93b-b5158d9421fb admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:55.666251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 996/1980] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:55 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:55.668756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:55.683087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bd29f-ddb0-41e6-a650-acfd54f3c28c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:55.778246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db5332-e644-4664-a348-ad20f2b81d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:55.779369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5332-e644-4664-a348-ad20f2b81d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.779920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5332-e644-4664-a348-ad20f2b81d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.780466 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5332-e644-4664-a348-ad20f2b81d33 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.781818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db5332-e644-4664-a348-ad20f2b81d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:55.787133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db5332-e644-4664-a348-ad20f2b81d33 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:55.828904 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 985/1981] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:55 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:55.833610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbeff3b-1d16-4b20-a93b-b5158d9421fb admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:55.847851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbeff3b-1d16-4b20-a93b-b5158d9421fb admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:55.856214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d16b60-26e8-40d9-8148-b62150f87869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:55.857722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d16b60-26e8-40d9-8148-b62150f87869 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.858564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d16b60-26e8-40d9-8148-b62150f87869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.858744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d16b60-26e8-40d9-8148-b62150f87869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.859835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d16b60-26e8-40d9-8148-b62150f87869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:55.863301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d16b60-26e8-40d9-8148-b62150f87869 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:55.884636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 997/1982] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:55.887646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db5332-e644-4664-a348-ad20f2b81d33 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:55.904710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db5332-e644-4664-a348-ad20f2b81d33 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:55.914322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b06d0-d627-4e66-b93d-08aa4a8d8007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:55.914961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b06d0-d627-4e66-b93d-08aa4a8d8007 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.915239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b06d0-d627-4e66-b93d-08aa4a8d8007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.915562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b06d0-d627-4e66-b93d-08aa4a8d8007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.916150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b06d0-d627-4e66-b93d-08aa4a8d8007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:55.917768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b06d0-d627-4e66-b93d-08aa4a8d8007 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:55.927098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:55.935246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 986/1983] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:55.939268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d16b60-26e8-40d9-8148-b62150f87869 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:55.950529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d16b60-26e8-40d9-8148-b62150f87869 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:55.955186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5575f90-c204-4fd5-87f1-2787604627de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:55.955836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575f90-c204-4fd5-87f1-2787604627de admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:55.956115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575f90-c204-4fd5-87f1-2787604627de admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:55.956389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575f90-c204-4fd5-87f1-2787604627de admin admin] PATH_INFO: `/v3/projects/a8d99217e4d4406b9dc6369001406c84/users/d85ffcebe8064f68ac8316d2412b6dc1/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:55.976215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5575f90-c204-4fd5-87f1-2787604627de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d99217e4d4406b9dc6369001406c84, user_id=d85ffcebe8064f68ac8316d2412b6dc1, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:55.979018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5575f90-c204-4fd5-87f1-2787604627de admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:55.994538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 998/1984] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:55 2022] PUT /identity/v3/projects/a8d99217e4d4406b9dc6369001406c84/users/d85ffcebe8064f68ac8316d2412b6dc1/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:56.005247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bbe2e-338e-4490-b880-220d62dde9f7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.005533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bbe2e-338e-4490-b880-220d62dde9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.005828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bbe2e-338e-4490-b880-220d62dde9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.042603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b83bbe2e-338e-4490-b880-220d62dde9f7 None None] MFA Rules not processed for user `d85ffcebe8064f68ac8316d2412b6dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:56.043852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b83bbe2e-338e-4490-b880-220d62dde9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:56.114652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 987/1985] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:56.119179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5575f90-c204-4fd5-87f1-2787604627de admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:56.191632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5575f90-c204-4fd5-87f1-2787604627de admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:56.195835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268c025-6f4e-4d6b-82c7-0946745307c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:56.196472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c025-6f4e-4d6b-82c7-0946745307c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.196769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c025-6f4e-4d6b-82c7-0946745307c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.197047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268c025-6f4e-4d6b-82c7-0946745307c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.197703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268c025-6f4e-4d6b-82c7-0946745307c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:56.199644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268c025-6f4e-4d6b-82c7-0946745307c0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:56.219062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:56.240054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 999/1986] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:18:56 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:56.244339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bbe2e-338e-4490-b880-220d62dde9f7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:56.266576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bbe2e-338e-4490-b880-220d62dde9f7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:56.340305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01040c2a-6757-492a-8e92-3fab99896e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:56.340949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01040c2a-6757-492a-8e92-3fab99896e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.341230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01040c2a-6757-492a-8e92-3fab99896e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.341494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01040c2a-6757-492a-8e92-3fab99896e8d admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.342131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01040c2a-6757-492a-8e92-3fab99896e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:56.344032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01040c2a-6757-492a-8e92-3fab99896e8d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:56.368047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 988/1987] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:18:56 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:18:56.371933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268c025-6f4e-4d6b-82c7-0946745307c0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:56.383081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268c025-6f4e-4d6b-82c7-0946745307c0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:56.387658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2651d34-5b32-4392-9d92-d50f8a74d9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:56.388252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2651d34-5b32-4392-9d92-d50f8a74d9ac admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.388530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2651d34-5b32-4392-9d92-d50f8a74d9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.388868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2651d34-5b32-4392-9d92-d50f8a74d9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.389517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2651d34-5b32-4392-9d92-d50f8a74d9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:56.391116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2651d34-5b32-4392-9d92-d50f8a74d9ac admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:56.403202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1000/1988] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:18:56.407376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01040c2a-6757-492a-8e92-3fab99896e8d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:56.418147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01040c2a-6757-492a-8e92-3fab99896e8d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:56.423337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460b41e-d8ae-428f-bfcf-c7e79660c5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:56.424016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b41e-d8ae-428f-bfcf-c7e79660c5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.424494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b41e-d8ae-428f-bfcf-c7e79660c5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.430189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b41e-d8ae-428f-bfcf-c7e79660c5ca admin admin] PATH_INFO: `/v3/projects/046ea4d2a71f4bd4b9a9da6c91397768/users/ce0ffa0de8b44a39ab6ca9028543349b/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.447878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460b41e-d8ae-428f-bfcf-c7e79660c5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=046ea4d2a71f4bd4b9a9da6c91397768, user_id=ce0ffa0de8b44a39ab6ca9028543349b, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:56.450675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460b41e-d8ae-428f-bfcf-c7e79660c5ca admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:56.465751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 989/1989] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:56 2022] PUT /identity/v3/projects/046ea4d2a71f4bd4b9a9da6c91397768/users/ce0ffa0de8b44a39ab6ca9028543349b/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:18:56.470073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2651d34-5b32-4392-9d92-d50f8a74d9ac admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:56.480733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2651d34-5b32-4392-9d92-d50f8a74d9ac admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:56.556506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683f8eb-5235-4423-b925-cd3df11fad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:56.557386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683f8eb-5235-4423-b925-cd3df11fad37 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.557824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683f8eb-5235-4423-b925-cd3df11fad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.558162 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683f8eb-5235-4423-b925-cd3df11fad37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.558999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683f8eb-5235-4423-b925-cd3df11fad37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:56.561924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683f8eb-5235-4423-b925-cd3df11fad37 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:56.576462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1001/1990] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:18:56.580812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460b41e-d8ae-428f-bfcf-c7e79660c5ca admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:56.593849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460b41e-d8ae-428f-bfcf-c7e79660c5ca admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:56.608628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:56.609577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.609961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.610334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] PATH_INFO: `/v3/projects/046ea4d2a71f4bd4b9a9da6c91397768/users/ce0ffa0de8b44a39ab6ca9028543349b/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.612691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=046ea4d2a71f4bd4b9a9da6c91397768, user_id=ce0ffa0de8b44a39ab6ca9028543349b, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:56.616380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:56.637712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 990/1991] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:18:56 2022] PUT /identity/v3/projects/046ea4d2a71f4bd4b9a9da6c91397768/users/ce0ffa0de8b44a39ab6ca9028543349b/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:18:56.650028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81821f92-b64e-47f6-8125-39bd01e51101 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.650324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81821f92-b64e-47f6-8125-39bd01e51101 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.650604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81821f92-b64e-47f6-8125-39bd01e51101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.695071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-81821f92-b64e-47f6-8125-39bd01e51101 None None] MFA Rules not processed for user `ce0ffa0de8b44a39ab6ca9028543349b`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:56.696481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-81821f92-b64e-47f6-8125-39bd01e51101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:56.785106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1002/1992] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:18:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:56.796548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:56.798366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:56.857987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:56.872262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e99547-63d8-4e8c-87da-f5711c0af997 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:56.886518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e2d6-5af9-4996-873c-8ccdc08c2df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:56.887214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e2d6-5af9-4996-873c-8ccdc08c2df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:56.887531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e2d6-5af9-4996-873c-8ccdc08c2df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:56.890463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7e2d6-5af9-4996-873c-8ccdc08c2df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:56.908882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e2d6-5af9-4996-873c-8ccdc08c2df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:56.911076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7e2d6-5af9-4996-873c-8ccdc08c2df5 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:56.922294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 991/1993] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:57.002140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81821f92-b64e-47f6-8125-39bd01e51101 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:57.012775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81821f92-b64e-47f6-8125-39bd01e51101 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:57.017281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c180d-a7d6-4ce6-b47f-78b2199e93c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:57.018010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c180d-a7d6-4ce6-b47f-78b2199e93c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:57.018270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c180d-a7d6-4ce6-b47f-78b2199e93c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:57.018546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c180d-a7d6-4ce6-b47f-78b2199e93c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:57.082911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c180d-a7d6-4ce6-b47f-78b2199e93c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:57.083942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c180d-a7d6-4ce6-b47f-78b2199e93c6 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:57.093279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1003/1994] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:06.917456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:07.088891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:INSERT=1,keystone:SELECT=98 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:08.267126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e2d6-5af9-4996-873c-8ccdc08c2df5 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:08.337516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7e2d6-5af9-4996-873c-8ccdc08c2df5 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:08.342196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:08.342854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:08.343135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:08.343448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] PATH_INFO: `/v3/users/d85ffcebe8064f68ac8316d2412b6dc1` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:08.344585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d85ffcebe8064f68ac8316d2412b6dc1)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:08.346716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:08.424325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd85ffcebe8064f68ac8316d2412b6dc1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:08.441244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd85ffcebe8064f68ac8316d2412b6dc1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:08.442322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd85ffcebe8064f68ac8316d2412b6dc1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:08.443299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd85ffcebe8064f68ac8316d2412b6dc1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:08.452201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd85ffcebe8064f68ac8316d2412b6dc1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:08.477739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 992/1995] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:08 2022] DELETE /identity/v3/users/d85ffcebe8064f68ac8316d2412b6dc1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:08.482213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40809ba0-dfe3-4318-8ada-01536b4069ae None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:08.482828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40809ba0-dfe3-4318-8ada-01536b4069ae None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:08.482828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40809ba0-dfe3-4318-8ada-01536b4069ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:08.504259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-40809ba0-dfe3-4318-8ada-01536b4069ae None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:08.506519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-40809ba0-dfe3-4318-8ada-01536b4069ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:08.576044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1004/1996] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:08.586072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:08.587496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:08.649908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f30e-3abb-479a-8ea7-746b0604b843 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:08.653470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b363d2-93e9-4490-8623-6b16e48b4790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:08.654101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b363d2-93e9-4490-8623-6b16e48b4790 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:08.654373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b363d2-93e9-4490-8623-6b16e48b4790 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:08.654638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b363d2-93e9-4490-8623-6b16e48b4790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:08.663724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b363d2-93e9-4490-8623-6b16e48b4790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:08.665184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b363d2-93e9-4490-8623-6b16e48b4790 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:08.674740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 993/1997] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:19:08.949166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40809ba0-dfe3-4318-8ada-01536b4069ae None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:08.950628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-40809ba0-dfe3-4318-8ada-01536b4069ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:08.962690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40809ba0-dfe3-4318-8ada-01536b4069ae None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:08.968036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:08.969440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:08.969948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:08.970238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] PATH_INFO: `/v3/projects/a8d99217e4d4406b9dc6369001406c84` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:08.971385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d99217e4d4406b9dc6369001406c84)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:08.974468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:09.025058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d99217e4d4406b9dc6369001406c84', 'request_id': 'req-3698e603-5391-4d21-9a8c-e5844df13627'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.031190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d99217e4d4406b9dc6369001406c84', 'request_id': 'req-3698e603-5391-4d21-9a8c-e5844df13627'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.037460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d99217e4d4406b9dc6369001406c84', 'request_id': 'req-3698e603-5391-4d21-9a8c-e5844df13627'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.057855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] The token cache is being invalidate because project a8d99217e4d4406b9dc6369001406c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:09.058160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.058834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.060357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1005/1998] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:08 2022] DELETE /identity/v3/projects/a8d99217e4d4406b9dc6369001406c84 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:09.064337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b363d2-93e9-4490-8623-6b16e48b4790 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:09.074838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b363d2-93e9-4490-8623-6b16e48b4790 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:09.093945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:09.140456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:09.141127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:09.141408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:09.141680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] PATH_INFO: `/v3/users/ce0ffa0de8b44a39ab6ca9028543349b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:09.143107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce0ffa0de8b44a39ab6ca9028543349b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:09.145684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:09.202768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce0ffa0de8b44a39ab6ca9028543349b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.218871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce0ffa0de8b44a39ab6ca9028543349b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.220973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce0ffa0de8b44a39ab6ca9028543349b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.221942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce0ffa0de8b44a39ab6ca9028543349b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.232122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25a7a8a-b6fe-49f2-9c38-8fd0be8022d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce0ffa0de8b44a39ab6ca9028543349b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.250937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 994/1999] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:09 2022] DELETE /identity/v3/users/ce0ffa0de8b44a39ab6ca9028543349b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:09.573418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:09.575469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:09.589421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e603-5391-4d21-9a8c-e5844df13627 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:09.653059 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:09.653705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:09.653984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:09.654256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] PATH_INFO: `/v3/projects/046ea4d2a71f4bd4b9a9da6c91397768` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:09.655648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=046ea4d2a71f4bd4b9a9da6c91397768)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:09.657686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:09.695484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:09.715210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046ea4d2a71f4bd4b9a9da6c91397768', 'request_id': 'req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.725488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046ea4d2a71f4bd4b9a9da6c91397768', 'request_id': 'req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.731749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046ea4d2a71f4bd4b9a9da6c91397768', 'request_id': 'req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.749221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] The token cache is being invalidate because project 046ea4d2a71f4bd4b9a9da6c91397768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:09.749914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.750614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-87fcc5c0-309f-49e4-9ffe-d6cfc4734043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:09.753500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1006/2000] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:09 2022] DELETE /identity/v3/projects/046ea4d2a71f4bd4b9a9da6c91397768 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:09.759458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9fa2-02ae-44b3-b436-00e910320ab3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:09.759995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9fa2-02ae-44b3-b436-00e910320ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:09.760774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff9fa2-02ae-44b3-b436-00e910320ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:09.784877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-15ff9fa2-02ae-44b3-b436-00e910320ab3 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:09.786616 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-15ff9fa2-02ae-44b3-b436-00e910320ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:09.852744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 995/2001] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:09.860592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771b21-f1ad-4f52-b4bd-da8dcb4e9ed1 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:09.861168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771b21-f1ad-4f52-b4bd-da8dcb4e9ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:09.861459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be771b21-f1ad-4f52-b4bd-da8dcb4e9ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:09.883600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-be771b21-f1ad-4f52-b4bd-da8dcb4e9ed1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:09.884455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-be771b21-f1ad-4f52-b4bd-da8dcb4e9ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:09.892670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1007/2002] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:19:09.897737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff9fa2-02ae-44b3-b436-00e910320ab3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:09.908338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff9fa2-02ae-44b3-b436-00e910320ab3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:09.915561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:09.916192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:09.916473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:09.916775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:09.917431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:19:09.917733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:09.919367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:09.936384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 996/2003] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:19:09.937550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be771b21-f1ad-4f52-b4bd-da8dcb4e9ed1 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:09.947609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be771b21-f1ad-4f52-b4bd-da8dcb4e9ed1 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:09.952346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa821bd-0e0d-444d-bdb3-f4d21e1318ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:09.952982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa821bd-0e0d-444d-bdb3-f4d21e1318ec admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:09.953262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa821bd-0e0d-444d-bdb3-f4d21e1318ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:09.953534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa821bd-0e0d-444d-bdb3-f4d21e1318ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:09.954177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa821bd-0e0d-444d-bdb3-f4d21e1318ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:09.956634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa821bd-0e0d-444d-bdb3-f4d21e1318ec admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:09.986366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1008/2004] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:09 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:19:09.990472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.001893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f73c42-f7d9-4f22-96d3-f0abb9d5bdc4 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.030307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:10.066252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76d0a2-cfdb-4eaa-a82a-ffd13d03c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.066896 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76d0a2-cfdb-4eaa-a82a-ffd13d03c27d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.067293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76d0a2-cfdb-4eaa-a82a-ffd13d03c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.067719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76d0a2-cfdb-4eaa-a82a-ffd13d03c27d admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.068344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76d0a2-cfdb-4eaa-a82a-ffd13d03c27d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.070635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76d0a2-cfdb-4eaa-a82a-ffd13d03c27d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.095794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 997/2005] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:09 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:19:10.100783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa821bd-0e0d-444d-bdb3-f4d21e1318ec admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.111281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa821bd-0e0d-444d-bdb3-f4d21e1318ec admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.116505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ccb11-ecf3-4963-a90d-15f34ae6b597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.117134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ccb11-ecf3-4963-a90d-15f34ae6b597 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.117412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ccb11-ecf3-4963-a90d-15f34ae6b597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.117680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ccb11-ecf3-4963-a90d-15f34ae6b597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.118322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ccb11-ecf3-4963-a90d-15f34ae6b597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.119980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ccb11-ecf3-4963-a90d-15f34ae6b597 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.130899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1009/2006] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:10.135090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76d0a2-cfdb-4eaa-a82a-ffd13d03c27d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.147112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76d0a2-cfdb-4eaa-a82a-ffd13d03c27d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.152312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ccf8e-37b6-436d-b3fa-b4dce16733dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.152962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ccf8e-37b6-436d-b3fa-b4dce16733dc admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.153242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ccf8e-37b6-436d-b3fa-b4dce16733dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.153509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ccf8e-37b6-436d-b3fa-b4dce16733dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.154154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ccf8e-37b6-436d-b3fa-b4dce16733dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.155778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ccf8e-37b6-436d-b3fa-b4dce16733dc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.167939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 998/2007] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:10.172350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ccb11-ecf3-4963-a90d-15f34ae6b597 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.183514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ccb11-ecf3-4963-a90d-15f34ae6b597 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.188280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4940b-2fff-47ef-9657-f6ddc60925bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.189127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4940b-2fff-47ef-9657-f6ddc60925bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.189406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4940b-2fff-47ef-9657-f6ddc60925bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.189678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4940b-2fff-47ef-9657-f6ddc60925bf admin admin] PATH_INFO: `/v3/projects/d2212b1f358d4a2890b689673b75bf4b/users/3b9efbe22a4240bda076c12f25af3019/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.206608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4940b-2fff-47ef-9657-f6ddc60925bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2212b1f358d4a2890b689673b75bf4b, user_id=3b9efbe22a4240bda076c12f25af3019, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.209620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4940b-2fff-47ef-9657-f6ddc60925bf admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.227411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1010/2008] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:10 2022] PUT /identity/v3/projects/d2212b1f358d4a2890b689673b75bf4b/users/3b9efbe22a4240bda076c12f25af3019/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:10.239119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc8d16-da9c-42ae-ac8c-cc984127e328 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.239403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc8d16-da9c-42ae-ac8c-cc984127e328 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.239679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc8d16-da9c-42ae-ac8c-cc984127e328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.271248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-12bc8d16-da9c-42ae-ac8c-cc984127e328 None None] MFA Rules not processed for user `3b9efbe22a4240bda076c12f25af3019`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:10.272542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-12bc8d16-da9c-42ae-ac8c-cc984127e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:10.342913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 999/2009] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:10.347990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4940b-2fff-47ef-9657-f6ddc60925bf admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.440099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4940b-2fff-47ef-9657-f6ddc60925bf admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.440099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da1cd-ee3d-4be8-98d0-468f03543dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.440099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da1cd-ee3d-4be8-98d0-468f03543dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.440099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da1cd-ee3d-4be8-98d0-468f03543dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.441389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da1cd-ee3d-4be8-98d0-468f03543dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.441389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214da1cd-ee3d-4be8-98d0-468f03543dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.442688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214da1cd-ee3d-4be8-98d0-468f03543dcb admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.479509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1011/2010] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:10 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:10.483627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc8d16-da9c-42ae-ac8c-cc984127e328 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.497117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc8d16-da9c-42ae-ac8c-cc984127e328 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.571078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c26c4-ae49-48dc-8f71-5b3ee52322f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.572125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c26c4-ae49-48dc-8f71-5b3ee52322f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.572472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c26c4-ae49-48dc-8f71-5b3ee52322f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.572785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c26c4-ae49-48dc-8f71-5b3ee52322f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.573434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c26c4-ae49-48dc-8f71-5b3ee52322f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.575512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c26c4-ae49-48dc-8f71-5b3ee52322f2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.611139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1000/2011] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:10 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:10.613170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da1cd-ee3d-4be8-98d0-468f03543dcb admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.624604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da1cd-ee3d-4be8-98d0-468f03543dcb admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.630671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e6800-a19a-4cd2-a9f9-0b6a39be5c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.631281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6800-a19a-4cd2-a9f9-0b6a39be5c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.631696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6800-a19a-4cd2-a9f9-0b6a39be5c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.632099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6800-a19a-4cd2-a9f9-0b6a39be5c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.632698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e6800-a19a-4cd2-a9f9-0b6a39be5c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.634394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e6800-a19a-4cd2-a9f9-0b6a39be5c75 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.648408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1012/2012] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:10.653379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c26c4-ae49-48dc-8f71-5b3ee52322f2 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.666016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c26c4-ae49-48dc-8f71-5b3ee52322f2 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.682000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7f8b9-025c-4e1f-b2c3-08e6c7c9fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.683170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7f8b9-025c-4e1f-b2c3-08e6c7c9fbfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.683463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7f8b9-025c-4e1f-b2c3-08e6c7c9fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.683794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7f8b9-025c-4e1f-b2c3-08e6c7c9fbfa admin admin] PATH_INFO: `/v3/projects/b955e417096c47d9a1c3343ed5b7e55d/users/fe391556ebc84a4e90a38f053acefaee/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.709456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7f8b9-025c-4e1f-b2c3-08e6c7c9fbfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b955e417096c47d9a1c3343ed5b7e55d, user_id=fe391556ebc84a4e90a38f053acefaee, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.714102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7f8b9-025c-4e1f-b2c3-08e6c7c9fbfa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.733666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1001/2013] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:10 2022] PUT /identity/v3/projects/b955e417096c47d9a1c3343ed5b7e55d/users/fe391556ebc84a4e90a38f053acefaee/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:10.739194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e6800-a19a-4cd2-a9f9-0b6a39be5c75 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.755098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e6800-a19a-4cd2-a9f9-0b6a39be5c75 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.780271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:10.825281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477b79b-cb77-4066-bc61-a04f16b9efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.825680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477b79b-cb77-4066-bc61-a04f16b9efbb admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.825970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477b79b-cb77-4066-bc61-a04f16b9efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.826212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477b79b-cb77-4066-bc61-a04f16b9efbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.826869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477b79b-cb77-4066-bc61-a04f16b9efbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.828487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477b79b-cb77-4066-bc61-a04f16b9efbb admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.841474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1013/2014] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:10.848964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7f8b9-025c-4e1f-b2c3-08e6c7c9fbfa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:10.855377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:10.861317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7f8b9-025c-4e1f-b2c3-08e6c7c9fbfa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:10.868396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:10.869105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.869381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.869655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] PATH_INFO: `/v3/projects/b955e417096c47d9a1c3343ed5b7e55d/users/fe391556ebc84a4e90a38f053acefaee/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.871629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b955e417096c47d9a1c3343ed5b7e55d, user_id=fe391556ebc84a4e90a38f053acefaee, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:10.874357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:10.887652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1002/2015] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:10 2022] PUT /identity/v3/projects/b955e417096c47d9a1c3343ed5b7e55d/users/fe391556ebc84a4e90a38f053acefaee/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:10.898073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb0623-ebea-4d42-bb5d-61a7a9cb294b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:10.898360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb0623-ebea-4d42-bb5d-61a7a9cb294b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:10.898639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb0623-ebea-4d42-bb5d-61a7a9cb294b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:10.930043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-eafb0623-ebea-4d42-bb5d-61a7a9cb294b None None] MFA Rules not processed for user `fe391556ebc84a4e90a38f053acefaee`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:10.931333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-eafb0623-ebea-4d42-bb5d-61a7a9cb294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:11.024600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1014/2016] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:11.037861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:11.040033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:11.108326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ccb0a-8c69-4a92-a585-34c94ed5c13e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:11.112192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b640c-4a4d-4e8f-b133-e474dafd7652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:11.113426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b640c-4a4d-4e8f-b133-e474dafd7652 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:11.113707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b640c-4a4d-4e8f-b133-e474dafd7652 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:11.113977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b640c-4a4d-4e8f-b133-e474dafd7652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:11.124384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b640c-4a4d-4e8f-b133-e474dafd7652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:11.128384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b640c-4a4d-4e8f-b133-e474dafd7652 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:11.137249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1003/2017] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:11.667962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb0623-ebea-4d42-bb5d-61a7a9cb294b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:11.761151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb0623-ebea-4d42-bb5d-61a7a9cb294b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:11.767837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:11.769062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:11.769357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:11.769623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] PATH_INFO: `/v3/users/3b9efbe22a4240bda076c12f25af3019` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:11.770794 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b9efbe22a4240bda076c12f25af3019)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:11.772778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:11.870428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b9efbe22a4240bda076c12f25af3019'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:11.871778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:11.880469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b9efbe22a4240bda076c12f25af3019'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:11.881335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b9efbe22a4240bda076c12f25af3019'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:11.897402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b9efbe22a4240bda076c12f25af3019'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:11.898136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b9efbe22a4240bda076c12f25af3019'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:11.912923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1015/2018] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:11 2022] DELETE /identity/v3/users/3b9efbe22a4240bda076c12f25af3019 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:19:11.917575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205e2c-b5eb-4d79-be53-a6f2c7b8a425 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:11.918072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205e2c-b5eb-4d79-be53-a6f2c7b8a425 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:11.918182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205e2c-b5eb-4d79-be53-a6f2c7b8a425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:11.938836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d7205e2c-b5eb-4d79-be53-a6f2c7b8a425 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:11.940908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d7205e2c-b5eb-4d79-be53-a6f2c7b8a425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:12.010526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1004/2019] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:12.021383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:12.023031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:12.094686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6ecde-1c71-490a-997c-11355a29046c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:12.098742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f7a3d-4527-4c08-adea-02a652166820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:12.099392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f7a3d-4527-4c08-adea-02a652166820 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:12.099681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f7a3d-4527-4c08-adea-02a652166820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:12.099944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f7a3d-4527-4c08-adea-02a652166820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:12.109858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f7a3d-4527-4c08-adea-02a652166820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:12.111531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f7a3d-4527-4c08-adea-02a652166820 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:12.122549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1016/2020] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:12.486875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7205e2c-b5eb-4d79-be53-a6f2c7b8a425 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:12.490665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d7205e2c-b5eb-4d79-be53-a6f2c7b8a425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:12.503701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7205e2c-b5eb-4d79-be53-a6f2c7b8a425 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:12.508339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:12.512348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:12.512944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:12.513325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] PATH_INFO: `/v3/projects/d2212b1f358d4a2890b689673b75bf4b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:12.514410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2212b1f358d4a2890b689673b75bf4b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:12.516344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:12.572329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2212b1f358d4a2890b689673b75bf4b', 'request_id': 'req-65100df1-ae43-4fe7-a57f-4aec84cd47a4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.583682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2212b1f358d4a2890b689673b75bf4b', 'request_id': 'req-65100df1-ae43-4fe7-a57f-4aec84cd47a4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.593777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2212b1f358d4a2890b689673b75bf4b', 'request_id': 'req-65100df1-ae43-4fe7-a57f-4aec84cd47a4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.607469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] The token cache is being invalidate because project d2212b1f358d4a2890b689673b75bf4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:12.607789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.608555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.610099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1005/2021] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:12 2022] DELETE /identity/v3/projects/d2212b1f358d4a2890b689673b75bf4b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:12.614321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f7a3d-4527-4c08-adea-02a652166820 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:12.624056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f7a3d-4527-4c08-adea-02a652166820 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:12.689714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:12.691048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:12.691732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:12.692022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] PATH_INFO: `/v3/users/fe391556ebc84a4e90a38f053acefaee` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:12.693201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe391556ebc84a4e90a38f053acefaee)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:12.695169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:12.753512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe391556ebc84a4e90a38f053acefaee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.760528 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe391556ebc84a4e90a38f053acefaee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.761220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe391556ebc84a4e90a38f053acefaee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.772948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:12.775413 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe391556ebc84a4e90a38f053acefaee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.776782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9f466e5e-82c5-4205-ac39-8633d24cfda6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe391556ebc84a4e90a38f053acefaee'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:12.794857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1017/2022] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:12 2022] DELETE /identity/v3/users/fe391556ebc84a4e90a38f053acefaee => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:13.330362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:13.332339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:13.350971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65100df1-ae43-4fe7-a57f-4aec84cd47a4 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:13.368330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:13.434417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:13.435080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.435486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.435486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] PATH_INFO: `/v3/projects/b955e417096c47d9a1c3343ed5b7e55d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.435486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b955e417096c47d9a1c3343ed5b7e55d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:13.439298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:13.510401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b955e417096c47d9a1c3343ed5b7e55d', 'request_id': 'req-cd34733b-3ca8-4787-82f9-04b4b61dacc9'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:13.519745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b955e417096c47d9a1c3343ed5b7e55d', 'request_id': 'req-cd34733b-3ca8-4787-82f9-04b4b61dacc9'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:13.529745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b955e417096c47d9a1c3343ed5b7e55d', 'request_id': 'req-cd34733b-3ca8-4787-82f9-04b4b61dacc9'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:13.542570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] The token cache is being invalidate because project b955e417096c47d9a1c3343ed5b7e55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:13.542934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:13.543559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cd34733b-3ca8-4787-82f9-04b4b61dacc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:13.545111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1006/2023] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:13 2022] DELETE /identity/v3/projects/b955e417096c47d9a1c3343ed5b7e55d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:13.549990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e66bcf-ecc8-4918-a924-182007986881 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.550356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e66bcf-ecc8-4918-a924-182007986881 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.550550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e66bcf-ecc8-4918-a924-182007986881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.573230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b8e66bcf-ecc8-4918-a924-182007986881 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:13.574514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b8e66bcf-ecc8-4918-a924-182007986881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:13.644738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1018/2024] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:13.652800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7f843-de98-46a0-9b53-c86f428ec30d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.653084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7f843-de98-46a0-9b53-c86f428ec30d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.653364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7f843-de98-46a0-9b53-c86f428ec30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.679268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4ac7f843-de98-46a0-9b53-c86f428ec30d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:13.680681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4ac7f843-de98-46a0-9b53-c86f428ec30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:13.689042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1007/2025] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:13.694119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e66bcf-ecc8-4918-a924-182007986881 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:13.705048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e66bcf-ecc8-4918-a924-182007986881 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:13.712827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:13.713451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.713758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.714012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.714711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:19:13.715016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:13.716893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:13.730569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1019/2026] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:19:13.734586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7f843-de98-46a0-9b53-c86f428ec30d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:13.747432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7f843-de98-46a0-9b53-c86f428ec30d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:13.755369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbcaa1-072d-4dfa-afa6-e6ed5d09d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:13.756517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbcaa1-072d-4dfa-afa6-e6ed5d09d5ca admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.757032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbcaa1-072d-4dfa-afa6-e6ed5d09d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.757835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbcaa1-072d-4dfa-afa6-e6ed5d09d5ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.759382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbcaa1-072d-4dfa-afa6-e6ed5d09d5ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:13.763218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbcaa1-072d-4dfa-afa6-e6ed5d09d5ca admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:13.807963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1008/2027] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:13 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:13.813679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:13.824332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422cbc4-b201-409e-9e75-26d06bc271c3 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:13.882788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9b7c4-c72c-4e0a-9dac-90f46e213ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:13.883406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9b7c4-c72c-4e0a-9dac-90f46e213ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.883685 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9b7c4-c72c-4e0a-9dac-90f46e213ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.883957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9b7c4-c72c-4e0a-9dac-90f46e213ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.884747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9b7c4-c72c-4e0a-9dac-90f46e213ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:13.886727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9b7c4-c72c-4e0a-9dac-90f46e213ab1 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:13.913862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1020/2028] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:13 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:13.918130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbcaa1-072d-4dfa-afa6-e6ed5d09d5ca admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:13.932239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbcaa1-072d-4dfa-afa6-e6ed5d09d5ca admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:13.938503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a83ed-613f-4894-a354-e00022a7b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:13.939117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a83ed-613f-4894-a354-e00022a7b96e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.939403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a83ed-613f-4894-a354-e00022a7b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.939679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a83ed-613f-4894-a354-e00022a7b96e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.941038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a83ed-613f-4894-a354-e00022a7b96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:13.942672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a83ed-613f-4894-a354-e00022a7b96e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:13.953804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1009/2029] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:13.958642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9b7c4-c72c-4e0a-9dac-90f46e213ab1 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:13.975624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9b7c4-c72c-4e0a-9dac-90f46e213ab1 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:13.980344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801533e8-06de-4438-b165-54b02bbbb1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:13.980968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801533e8-06de-4438-b165-54b02bbbb1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:13.981243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801533e8-06de-4438-b165-54b02bbbb1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:13.981513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801533e8-06de-4438-b165-54b02bbbb1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:13.982149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801533e8-06de-4438-b165-54b02bbbb1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:13.983759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801533e8-06de-4438-b165-54b02bbbb1c8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:13.995817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1021/2030] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:14.000077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a83ed-613f-4894-a354-e00022a7b96e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:14.011153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a83ed-613f-4894-a354-e00022a7b96e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:14.015844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:14.016497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:14.016809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:14.017087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] PATH_INFO: `/v3/projects/593aa5caf3e54b6881f5c3933c9b23f0/users/fe9c401b03f54504a4f61684d4275d35/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:14.032254 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=593aa5caf3e54b6881f5c3933c9b23f0, user_id=fe9c401b03f54504a4f61684d4275d35, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:14.035075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:14.047961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1010/2031] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:13 2022] PUT /identity/v3/projects/593aa5caf3e54b6881f5c3933c9b23f0/users/fe9c401b03f54504a4f61684d4275d35/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:14.062394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314a2f8-8b4a-411e-bbcd-1a925bb9e1fd None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:14.062773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314a2f8-8b4a-411e-bbcd-1a925bb9e1fd None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:14.063124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314a2f8-8b4a-411e-bbcd-1a925bb9e1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:14.100760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f314a2f8-8b4a-411e-bbcd-1a925bb9e1fd None None] MFA Rules not processed for user `fe9c401b03f54504a4f61684d4275d35`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:14.102046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f314a2f8-8b4a-411e-bbcd-1a925bb9e1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:14.162247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:14.200799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1022/2032] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:14.212848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:14.214582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:14.275916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:14.289584 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1e234-e1b5-4648-a501-5d99e8a3e498 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:14.297033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196881c4-7684-4ca5-b6f7-103ba65d851d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:14.297975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196881c4-7684-4ca5-b6f7-103ba65d851d service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:14.298386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196881c4-7684-4ca5-b6f7-103ba65d851d service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:14.298794 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196881c4-7684-4ca5-b6f7-103ba65d851d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:14.308422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196881c4-7684-4ca5-b6f7-103ba65d851d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:14.309958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196881c4-7684-4ca5-b6f7-103ba65d851d service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:14.320736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1011/2033] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:14.343238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314a2f8-8b4a-411e-bbcd-1a925bb9e1fd None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:14.416326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314a2f8-8b4a-411e-bbcd-1a925bb9e1fd None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:14.426512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:14.427495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:14.427495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:14.428386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] PATH_INFO: `/v3/users/fe9c401b03f54504a4f61684d4275d35` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:14.430478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9c401b03f54504a4f61684d4275d35)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:14.434172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:14.484801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9c401b03f54504a4f61684d4275d35'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:14.493749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9c401b03f54504a4f61684d4275d35'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:14.494855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9c401b03f54504a4f61684d4275d35'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:14.507639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9c401b03f54504a4f61684d4275d35'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:14.508279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9c401b03f54504a4f61684d4275d35'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:14.526116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1023/2034] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:14 2022] DELETE /identity/v3/users/fe9c401b03f54504a4f61684d4275d35 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:14.531744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0151-45d0-4600-97a3-9fbda7022293 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:14.532191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0151-45d0-4600-97a3-9fbda7022293 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:14.532526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0151-45d0-4600-97a3-9fbda7022293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:14.554472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d5dc0151-45d0-4600-97a3-9fbda7022293 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:14.556445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d5dc0151-45d0-4600-97a3-9fbda7022293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:14.624078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1012/2035] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:14.635158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:14.636800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:14.702320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfee32-42a0-4ac5-9c7f-913a37fe4f0d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:14.707178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2a396-6e34-4612-8134-43dbf9de0e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:14.707804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2a396-6e34-4612-8134-43dbf9de0e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:14.708102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2a396-6e34-4612-8134-43dbf9de0e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:14.708375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2a396-6e34-4612-8134-43dbf9de0e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:14.724914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2a396-6e34-4612-8134-43dbf9de0e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:14.727187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2a396-6e34-4612-8134-43dbf9de0e3b service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:14.738961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1024/2036] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:15.071013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0151-45d0-4600-97a3-9fbda7022293 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:15.073001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d5dc0151-45d0-4600-97a3-9fbda7022293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:15.094974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0151-45d0-4600-97a3-9fbda7022293 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:15.109097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:15.109960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.110085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.110326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] PATH_INFO: `/v3/projects/593aa5caf3e54b6881f5c3933c9b23f0` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.111487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=593aa5caf3e54b6881f5c3933c9b23f0)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:15.113532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:15.172496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '593aa5caf3e54b6881f5c3933c9b23f0', 'request_id': 'req-02c1e49d-00bd-470e-a732-a5f395283c54'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:15.181278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '593aa5caf3e54b6881f5c3933c9b23f0', 'request_id': 'req-02c1e49d-00bd-470e-a732-a5f395283c54'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:15.189305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '593aa5caf3e54b6881f5c3933c9b23f0', 'request_id': 'req-02c1e49d-00bd-470e-a732-a5f395283c54'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:15.200837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] The token cache is being invalidate because project 593aa5caf3e54b6881f5c3933c9b23f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:15.201124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:15.201803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-02c1e49d-00bd-470e-a732-a5f395283c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:15.203333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1013/2037] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:15 2022] DELETE /identity/v3/projects/593aa5caf3e54b6881f5c3933c9b23f0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:15.211179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258b9db-b00f-47cf-8b54-8fc6ee73f7dc None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.211437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258b9db-b00f-47cf-8b54-8fc6ee73f7dc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.211890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258b9db-b00f-47cf-8b54-8fc6ee73f7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.220332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:15.233044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d258b9db-b00f-47cf-8b54-8fc6ee73f7dc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:15.234628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d258b9db-b00f-47cf-8b54-8fc6ee73f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:15.311004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1025/2038] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:15.320339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebd31d-f7a2-49cc-bbd4-eedf6906799b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.320735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebd31d-f7a2-49cc-bbd4-eedf6906799b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.321073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebd31d-f7a2-49cc-bbd4-eedf6906799b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.342600 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8aebd31d-f7a2-49cc-bbd4-eedf6906799b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:15.343922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8aebd31d-f7a2-49cc-bbd4-eedf6906799b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:15.353673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1014/2039] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:15.358151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258b9db-b00f-47cf-8b54-8fc6ee73f7dc None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:15.369883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258b9db-b00f-47cf-8b54-8fc6ee73f7dc None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:15.374548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:15.375150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.375902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.376174 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.376862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:19:15.377162 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:15.380296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:15.394487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1026/2040] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:19:15.398358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebd31d-f7a2-49cc-bbd4-eedf6906799b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:15.408220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebd31d-f7a2-49cc-bbd4-eedf6906799b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:15.413946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02412b-1414-49b6-91ca-b611e597463e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:15.414574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02412b-1414-49b6-91ca-b611e597463e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.414861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02412b-1414-49b6-91ca-b611e597463e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.415134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02412b-1414-49b6-91ca-b611e597463e admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.415788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02412b-1414-49b6-91ca-b611e597463e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:15.417772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02412b-1414-49b6-91ca-b611e597463e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:15.458852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1015/2041] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:15 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:15.463351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:15.474752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e746a3-3f19-4510-aaa6-cbcfa2cc4d10 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:15.559541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25858cf5-c554-4533-9ab2-d6bb07c6f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:15.560437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858cf5-c554-4533-9ab2-d6bb07c6f2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.560899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858cf5-c554-4533-9ab2-d6bb07c6f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.561316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25858cf5-c554-4533-9ab2-d6bb07c6f2b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.562309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858cf5-c554-4533-9ab2-d6bb07c6f2b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:15.566022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25858cf5-c554-4533-9ab2-d6bb07c6f2b4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:15.593692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1027/2042] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:15 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:19:15.598927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02412b-1414-49b6-91ca-b611e597463e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:15.613024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02412b-1414-49b6-91ca-b611e597463e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:15.618283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469580a0-5f77-4d6f-aa04-9c54eeff39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:15.618962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469580a0-5f77-4d6f-aa04-9c54eeff39f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.619314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469580a0-5f77-4d6f-aa04-9c54eeff39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.619649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469580a0-5f77-4d6f-aa04-9c54eeff39f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.620364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469580a0-5f77-4d6f-aa04-9c54eeff39f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:15.622137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469580a0-5f77-4d6f-aa04-9c54eeff39f7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:15.639929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1016/2043] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:15.640339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25858cf5-c554-4533-9ab2-d6bb07c6f2b4 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:15.653734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25858cf5-c554-4533-9ab2-d6bb07c6f2b4 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:15.664661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20426ca7-ae66-4c55-940a-15108683b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:15.665308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20426ca7-ae66-4c55-940a-15108683b00d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.665582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20426ca7-ae66-4c55-940a-15108683b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.665926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20426ca7-ae66-4c55-940a-15108683b00d admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.666634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20426ca7-ae66-4c55-940a-15108683b00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:15.668300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20426ca7-ae66-4c55-940a-15108683b00d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:15.683903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1028/2044] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:15.692342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469580a0-5f77-4d6f-aa04-9c54eeff39f7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:15.706639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469580a0-5f77-4d6f-aa04-9c54eeff39f7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:15.711752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:15.712423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.712739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.713014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] PATH_INFO: `/v3/projects/0ebfb6d4850046f78352f81dcc8f6660/users/83035f93b62146d4bcb34e8ef4be5ab4/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.733775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebfb6d4850046f78352f81dcc8f6660, user_id=83035f93b62146d4bcb34e8ef4be5ab4, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:15.736453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:15.752455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1017/2045] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:15 2022] PUT /identity/v3/projects/0ebfb6d4850046f78352f81dcc8f6660/users/83035f93b62146d4bcb34e8ef4be5ab4/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:15.763593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6b53e-ef87-4bd1-92c0-57b0bef4682d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.764452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6b53e-ef87-4bd1-92c0-57b0bef4682d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.764452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6b53e-ef87-4bd1-92c0-57b0bef4682d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.796722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-9ce6b53e-ef87-4bd1-92c0-57b0bef4682d None None] MFA Rules not processed for user `83035f93b62146d4bcb34e8ef4be5ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:15.798886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9ce6b53e-ef87-4bd1-92c0-57b0bef4682d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:15.873150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:15.878983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1029/2046] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:15.890901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:15.892820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:15.898532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:15.983393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45667dc6-be3f-42ed-99e0-061abe00246f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:15.987485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebfe5-f6d7-4861-9ac3-02b894bfe21b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6c90a2b95c498494a73b43e7138905', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:15.988057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebfe5-f6d7-4861-9ac3-02b894bfe21b service glance] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:15.988339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebfe5-f6d7-4861-9ac3-02b894bfe21b service glance] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:15.988638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebfe5-f6d7-4861-9ac3-02b894bfe21b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:15.999191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebfe5-f6d7-4861-9ac3-02b894bfe21b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:16.001099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebfe5-f6d7-4861-9ac3-02b894bfe21b service glance] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:16.015853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1018/2047] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:24.364206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2027f-6620-4a50-8adb-1afdd537e3c0 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:24.365135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2027f-6620-4a50-8adb-1afdd537e3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:24.365260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2027f-6620-4a50-8adb-1afdd537e3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:24.392437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-93e2027f-6620-4a50-8adb-1afdd537e3c0 None None] MFA Rules not processed for user `b7c7e84a2809475cb7fbfaf905c2ad5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:24.394013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-93e2027f-6620-4a50-8adb-1afdd537e3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:24.468683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1030/2048] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:26.007518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=33 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:30.809808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d537c-3747-4443-8e01-4e9559315847 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:30.810451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d537c-3747-4443-8e01-4e9559315847 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:30.810739 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d537c-3747-4443-8e01-4e9559315847 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:30.812796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1019/2049] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:30.822459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508485f-4bd7-46db-95e8-f8c0576de805 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:30.823009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508485f-4bd7-46db-95e8-f8c0576de805 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:30.823343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508485f-4bd7-46db-95e8-f8c0576de805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:30.856051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8508485f-4bd7-46db-95e8-f8c0576de805 None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:30.857457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8508485f-4bd7-46db-95e8-f8c0576de805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:30.894413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d537c-3747-4443-8e01-4e9559315847 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:30.898640 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e84d537c-3747-4443-8e01-4e9559315847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:30.986194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1031/2050] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:19:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:31.000384 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508485f-4bd7-46db-95e8-f8c0576de805 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:31.002790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8508485f-4bd7-46db-95e8-f8c0576de805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:31.021256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508485f-4bd7-46db-95e8-f8c0576de805 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:31.027135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2859f6-551b-4a03-9e85-c261887ecc83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:31.027813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2859f6-551b-4a03-9e85-c261887ecc83 service placement] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:31.028095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2859f6-551b-4a03-9e85-c261887ecc83 service placement] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:31.028420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2859f6-551b-4a03-9e85-c261887ecc83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:31.042996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2859f6-551b-4a03-9e85-c261887ecc83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:31.042996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2859f6-551b-4a03-9e85-c261887ecc83 service placement] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:31.047518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d537c-3747-4443-8e01-4e9559315847 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:31.054671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:31.055740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:31.056263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:31.056973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] PATH_INFO: `/v3/users/97c9201758884fdbb080f85337a5ce87` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:31.059575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c9201758884fdbb080f85337a5ce87)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:31.061243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1032/2051] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:31.063541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:31.137626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c9201758884fdbb080f85337a5ce87'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.157929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c9201758884fdbb080f85337a5ce87'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.160609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c9201758884fdbb080f85337a5ce87'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.160791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c9201758884fdbb080f85337a5ce87'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.173117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-617d5529-035d-4b76-a9c1-8b8d54b75c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c9201758884fdbb080f85337a5ce87'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.190699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1020/2052] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:30 2022] DELETE /identity/v3/users/97c9201758884fdbb080f85337a5ce87 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:31.476953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2859f6-551b-4a03-9e85-c261887ecc83 service placement] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:31.494468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2859f6-551b-4a03-9e85-c261887ecc83 service placement] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:31.560103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:31.560761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:31.561039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:31.561306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] PATH_INFO: `/v3/projects/9ccec772d24645ce8f5e58de8ee5e47c` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:31.562406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ccec772d24645ce8f5e58de8ee5e47c)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:31.564315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:31.601932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:31.618197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ccec772d24645ce8f5e58de8ee5e47c', 'request_id': 'req-38ce291f-0ec3-4898-8c37-b647662c3d7c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.627193 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ccec772d24645ce8f5e58de8ee5e47c', 'request_id': 'req-38ce291f-0ec3-4898-8c37-b647662c3d7c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.636337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ccec772d24645ce8f5e58de8ee5e47c', 'request_id': 'req-38ce291f-0ec3-4898-8c37-b647662c3d7c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.649780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] The token cache is being invalidate because project 9ccec772d24645ce8f5e58de8ee5e47c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:31.650390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.651022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-38ce291f-0ec3-4898-8c37-b647662c3d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:31.652883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1033/2053] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:31 2022] DELETE /identity/v3/projects/9ccec772d24645ce8f5e58de8ee5e47c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:31.658402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadab4a-b422-4967-b4a7-834beacc7d49 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:31.659258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadab4a-b422-4967-b4a7-834beacc7d49 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:31.659616 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadab4a-b422-4967-b4a7-834beacc7d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:31.681915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dcadab4a-b422-4967-b4a7-834beacc7d49 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:31.683168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dcadab4a-b422-4967-b4a7-834beacc7d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:31.750143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1021/2054] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:31.759769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e1fb-fd70-4827-8f43-a671f88f061f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:31.760361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e1fb-fd70-4827-8f43-a671f88f061f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:31.760971 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e1fb-fd70-4827-8f43-a671f88f061f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:31.786000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-3b02e1fb-fd70-4827-8f43-a671f88f061f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:31.787431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3b02e1fb-fd70-4827-8f43-a671f88f061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:31.799033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1034/2055] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:31.804323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadab4a-b422-4967-b4a7-834beacc7d49 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:31.814472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadab4a-b422-4967-b4a7-834beacc7d49 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:31.819502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:31.820110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:31.820383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:31.820677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:31.821328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:19:31.821627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:31.823264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:31.840242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1022/2056] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:19:31.841101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02e1fb-fd70-4827-8f43-a671f88f061f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:31.851270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02e1fb-fd70-4827-8f43-a671f88f061f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:31.855920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e8c4f-ae57-46cf-b514-640dba835201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:31.856517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e8c4f-ae57-46cf-b514-640dba835201 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:31.856828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e8c4f-ae57-46cf-b514-640dba835201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:31.857101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e8c4f-ae57-46cf-b514-640dba835201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:31.857745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e8c4f-ae57-46cf-b514-640dba835201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:31.859863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e8c4f-ae57-46cf-b514-640dba835201 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:31.895304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1035/2057] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:31 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:31.896772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:31.907008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1da17e-dba2-40bf-8702-d4336a409e12 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:31.913737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:32.003866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86951a33-caf1-416f-aca6-78e45fcf535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.005063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86951a33-caf1-416f-aca6-78e45fcf535b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.005349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86951a33-caf1-416f-aca6-78e45fcf535b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.005627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86951a33-caf1-416f-aca6-78e45fcf535b admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.006286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86951a33-caf1-416f-aca6-78e45fcf535b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.008868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86951a33-caf1-416f-aca6-78e45fcf535b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.032948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1023/2058] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:31 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:32.037667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e8c4f-ae57-46cf-b514-640dba835201 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.048673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e8c4f-ae57-46cf-b514-640dba835201 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.053788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6af89c-b986-41f3-b91a-44bb4dd30fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.054399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af89c-b986-41f3-b91a-44bb4dd30fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.054722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af89c-b986-41f3-b91a-44bb4dd30fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.054997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6af89c-b986-41f3-b91a-44bb4dd30fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.055644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6af89c-b986-41f3-b91a-44bb4dd30fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.057491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6af89c-b986-41f3-b91a-44bb4dd30fe0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.073401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1036/2059] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:19:32.074920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86951a33-caf1-416f-aca6-78e45fcf535b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.087003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86951a33-caf1-416f-aca6-78e45fcf535b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.091774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66f267-9c20-4a32-94b6-89c4a131bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.092389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f267-9c20-4a32-94b6-89c4a131bc51 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.092708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f267-9c20-4a32-94b6-89c4a131bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.092980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f267-9c20-4a32-94b6-89c4a131bc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.093623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66f267-9c20-4a32-94b6-89c4a131bc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.095331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66f267-9c20-4a32-94b6-89c4a131bc51 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.111627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1024/2060] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:32.113489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6af89c-b986-41f3-b91a-44bb4dd30fe0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.133126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6af89c-b986-41f3-b91a-44bb4dd30fe0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.138868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8b209-88c8-458f-a834-d5d19ad8de32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.139616 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b209-88c8-458f-a834-d5d19ad8de32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.140061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b209-88c8-458f-a834-d5d19ad8de32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.140373 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b209-88c8-458f-a834-d5d19ad8de32 admin admin] PATH_INFO: `/v3/projects/0d8de58eebfb47cea3822ead09d8ccf6/users/7cc8e7d89ec243e4822e44dcc3857781/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.157377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8b209-88c8-458f-a834-d5d19ad8de32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8de58eebfb47cea3822ead09d8ccf6, user_id=7cc8e7d89ec243e4822e44dcc3857781, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.160116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8b209-88c8-458f-a834-d5d19ad8de32 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.175265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1037/2061] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:32 2022] PUT /identity/v3/projects/0d8de58eebfb47cea3822ead09d8ccf6/users/7cc8e7d89ec243e4822e44dcc3857781/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:32.189467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5192f09-409a-4eb8-9116-4d93e346c0e5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.190318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5192f09-409a-4eb8-9116-4d93e346c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.190599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5192f09-409a-4eb8-9116-4d93e346c0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.229066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a5192f09-409a-4eb8-9116-4d93e346c0e5 None None] MFA Rules not processed for user `7cc8e7d89ec243e4822e44dcc3857781`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:32.230325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a5192f09-409a-4eb8-9116-4d93e346c0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:32.313062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1025/2062] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:32.318001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8b209-88c8-458f-a834-d5d19ad8de32 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.392886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8b209-88c8-458f-a834-d5d19ad8de32 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.403746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a6fe6-f04b-4087-8f52-b832b9471b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.405309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a6fe6-f04b-4087-8f52-b832b9471b20 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.406710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a6fe6-f04b-4087-8f52-b832b9471b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.407875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a6fe6-f04b-4087-8f52-b832b9471b20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.409340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a6fe6-f04b-4087-8f52-b832b9471b20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.411721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a6fe6-f04b-4087-8f52-b832b9471b20 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.443546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1038/2063] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:32 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:19:32.447409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5192f09-409a-4eb8-9116-4d93e346c0e5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.460690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5192f09-409a-4eb8-9116-4d93e346c0e5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.559610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8efff-1d39-4d84-a8b1-0d1164fe4e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.560433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8efff-1d39-4d84-a8b1-0d1164fe4e1b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.560863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8efff-1d39-4d84-a8b1-0d1164fe4e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.561181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8efff-1d39-4d84-a8b1-0d1164fe4e1b admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.561950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8efff-1d39-4d84-a8b1-0d1164fe4e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.563988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8efff-1d39-4d84-a8b1-0d1164fe4e1b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.607634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1026/2064] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:32 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:32.612325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a6fe6-f04b-4087-8f52-b832b9471b20 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.623889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a6fe6-f04b-4087-8f52-b832b9471b20 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.632007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d2dee-9cac-4b80-a491-0915feb05d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.632677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2dee-9cac-4b80-a491-0915feb05d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.632959 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2dee-9cac-4b80-a491-0915feb05d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.633640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d2dee-9cac-4b80-a491-0915feb05d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.634187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d2dee-9cac-4b80-a491-0915feb05d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.635871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d2dee-9cac-4b80-a491-0915feb05d12 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.649408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1039/2065] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:32.653661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8efff-1d39-4d84-a8b1-0d1164fe4e1b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.661168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:32.666017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8efff-1d39-4d84-a8b1-0d1164fe4e1b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.673194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617eb23-d150-4078-ad80-1aa3ad409e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.673816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617eb23-d150-4078-ad80-1aa3ad409e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.674092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617eb23-d150-4078-ad80-1aa3ad409e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.675045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617eb23-d150-4078-ad80-1aa3ad409e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.675783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617eb23-d150-4078-ad80-1aa3ad409e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.679557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617eb23-d150-4078-ad80-1aa3ad409e0e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.695308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1027/2066] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:32.696517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d2dee-9cac-4b80-a491-0915feb05d12 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.707613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d2dee-9cac-4b80-a491-0915feb05d12 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.712430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050826-5c71-4f8c-a8e6-ee0451672f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.713126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050826-5c71-4f8c-a8e6-ee0451672f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.713404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050826-5c71-4f8c-a8e6-ee0451672f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.713819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050826-5c71-4f8c-a8e6-ee0451672f92 admin admin] PATH_INFO: `/v3/projects/2fbeca86c8de411c8eb47d24be7abb69/users/dc2285e297f44393819d47c4f8473c46/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.733167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050826-5c71-4f8c-a8e6-ee0451672f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbeca86c8de411c8eb47d24be7abb69, user_id=dc2285e297f44393819d47c4f8473c46, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.736345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050826-5c71-4f8c-a8e6-ee0451672f92 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:32.760744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1040/2067] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:32 2022] PUT /identity/v3/projects/2fbeca86c8de411c8eb47d24be7abb69/users/dc2285e297f44393819d47c4f8473c46/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:19:32.773933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b352a9-2a1a-458b-803c-0efa70ebf162 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.774555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b352a9-2a1a-458b-803c-0efa70ebf162 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.775081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b352a9-2a1a-458b-803c-0efa70ebf162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.806921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-03b352a9-2a1a-458b-803c-0efa70ebf162 None None] MFA Rules not processed for user `dc2285e297f44393819d47c4f8473c46`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:32.808060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-03b352a9-2a1a-458b-803c-0efa70ebf162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:32.887050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1028/2068] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:32.891642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050826-5c71-4f8c-a8e6-ee0451672f92 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:32.901618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:32.969219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050826-5c71-4f8c-a8e6-ee0451672f92 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:32.976406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c2570-5a11-4365-aa35-34061aad547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:32.977201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c2570-5a11-4365-aa35-34061aad547f admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:32.977550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c2570-5a11-4365-aa35-34061aad547f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:32.977888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c2570-5a11-4365-aa35-34061aad547f admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:32.978611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c2570-5a11-4365-aa35-34061aad547f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:32.980658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c2570-5a11-4365-aa35-34061aad547f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:33.017894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1041/2069] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:32 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:33.019388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b352a9-2a1a-458b-803c-0efa70ebf162 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:33.031101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b352a9-2a1a-458b-803c-0efa70ebf162 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:33.112844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2273b21-57b0-45bb-b556-b17ce8447b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:33.113515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2273b21-57b0-45bb-b556-b17ce8447b77 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.113797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2273b21-57b0-45bb-b556-b17ce8447b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.114066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2273b21-57b0-45bb-b556-b17ce8447b77 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.114707 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2273b21-57b0-45bb-b556-b17ce8447b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:33.116667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2273b21-57b0-45bb-b556-b17ce8447b77 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:33.151401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1029/2070] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:33 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:33.157711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c2570-5a11-4365-aa35-34061aad547f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:33.176828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c2570-5a11-4365-aa35-34061aad547f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:33.184527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289a444-277f-4ffa-b782-56c4e6e9bec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:33.185327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289a444-277f-4ffa-b782-56c4e6e9bec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.185603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289a444-277f-4ffa-b782-56c4e6e9bec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.185891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289a444-277f-4ffa-b782-56c4e6e9bec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.186573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289a444-277f-4ffa-b782-56c4e6e9bec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:33.188562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289a444-277f-4ffa-b782-56c4e6e9bec7 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:33.202001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1042/2071] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:33.207441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2273b21-57b0-45bb-b556-b17ce8447b77 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:33.218413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2273b21-57b0-45bb-b556-b17ce8447b77 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:33.223679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c404c-1f8a-40ce-a75e-51e20d9f9204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:33.224341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c404c-1f8a-40ce-a75e-51e20d9f9204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.224646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c404c-1f8a-40ce-a75e-51e20d9f9204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.224922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c404c-1f8a-40ce-a75e-51e20d9f9204 admin admin] PATH_INFO: `/v3/projects/488ba340a2aa494e9b4a57febde6ac3d/users/632c198b30ea4701922fdd0e0381cce9/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.253985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c404c-1f8a-40ce-a75e-51e20d9f9204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488ba340a2aa494e9b4a57febde6ac3d, user_id=632c198b30ea4701922fdd0e0381cce9, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:33.258354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c404c-1f8a-40ce-a75e-51e20d9f9204 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:33.278250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1030/2072] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:33 2022] PUT /identity/v3/projects/488ba340a2aa494e9b4a57febde6ac3d/users/632c198b30ea4701922fdd0e0381cce9/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:33.282386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289a444-277f-4ffa-b782-56c4e6e9bec7 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:33.294127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289a444-277f-4ffa-b782-56c4e6e9bec7 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:33.356535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928044c-510e-488d-848b-aa597ff75869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:33.357170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928044c-510e-488d-848b-aa597ff75869 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.357449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928044c-510e-488d-848b-aa597ff75869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.357720 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928044c-510e-488d-848b-aa597ff75869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.358370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928044c-510e-488d-848b-aa597ff75869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:33.360013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928044c-510e-488d-848b-aa597ff75869 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:33.373688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1043/2073] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:33.374874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c404c-1f8a-40ce-a75e-51e20d9f9204 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:33.386390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c404c-1f8a-40ce-a75e-51e20d9f9204 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:33.393824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:33.394456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.394576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.394844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] PATH_INFO: `/v3/projects/488ba340a2aa494e9b4a57febde6ac3d/users/632c198b30ea4701922fdd0e0381cce9/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.398224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488ba340a2aa494e9b4a57febde6ac3d, user_id=632c198b30ea4701922fdd0e0381cce9, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:33.401185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:33.420466 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1031/2074] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:33 2022] PUT /identity/v3/projects/488ba340a2aa494e9b4a57febde6ac3d/users/632c198b30ea4701922fdd0e0381cce9/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:33.430509 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4999dd-1689-4bd9-86f5-1e3b6c1f8ba0 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.430800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4999dd-1689-4bd9-86f5-1e3b6c1f8ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.431255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4999dd-1689-4bd9-86f5-1e3b6c1f8ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.465563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-5e4999dd-1689-4bd9-86f5-1e3b6c1f8ba0 None None] MFA Rules not processed for user `632c198b30ea4701922fdd0e0381cce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:33.467037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5e4999dd-1689-4bd9-86f5-1e3b6c1f8ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:33.558088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:33.568674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1044/2075] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:33.582636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:33.584758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:33.643432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:33.668030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a11f3-c6ee-4e7b-9d7c-7d1cef29a011 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:33.671783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f1139-2226-4b6f-9817-87ea6f0b4750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:33.672404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f1139-2226-4b6f-9817-87ea6f0b4750 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:33.672716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f1139-2226-4b6f-9817-87ea6f0b4750 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:33.673074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f1139-2226-4b6f-9817-87ea6f0b4750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:33.684839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f1139-2226-4b6f-9817-87ea6f0b4750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:33.686511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f1139-2226-4b6f-9817-87ea6f0b4750 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:33.699942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1032/2076] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:34.342952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4999dd-1689-4bd9-86f5-1e3b6c1f8ba0 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:34.344598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5e4999dd-1689-4bd9-86f5-1e3b6c1f8ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:34.357938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4999dd-1689-4bd9-86f5-1e3b6c1f8ba0 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:34.362582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f704ae-5459-4f86-9463-38488b85d3f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:34.363966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f704ae-5459-4f86-9463-38488b85d3f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:34.363966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f704ae-5459-4f86-9463-38488b85d3f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:34.363966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f704ae-5459-4f86-9463-38488b85d3f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:34.462753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f704ae-5459-4f86-9463-38488b85d3f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:34.464323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f704ae-5459-4f86-9463-38488b85d3f1 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:34.477708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1045/2077] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:38.665186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f1139-2226-4b6f-9817-87ea6f0b4750 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:38.674599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f1139-2226-4b6f-9817-87ea6f0b4750 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:38.678989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bf357-92c4-4332-9f4d-4f4db5ab8e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:38.679588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bf357-92c4-4332-9f4d-4f4db5ab8e0d service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:38.679867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bf357-92c4-4332-9f4d-4f4db5ab8e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:38.680136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bf357-92c4-4332-9f4d-4f4db5ab8e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:38.761052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bf357-92c4-4332-9f4d-4f4db5ab8e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:38.763639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bf357-92c4-4332-9f4d-4f4db5ab8e0d service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:38.778805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1033/2078] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:41.049513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f704ae-5459-4f86-9463-38488b85d3f1 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:41.138323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f704ae-5459-4f86-9463-38488b85d3f1 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:41.144946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:41.145743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:41.146024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:41.146297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] PATH_INFO: `/v3/users/7cc8e7d89ec243e4822e44dcc3857781` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:41.147474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc8e7d89ec243e4822e44dcc3857781)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:41.149849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:41.212261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc8e7d89ec243e4822e44dcc3857781'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.221159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc8e7d89ec243e4822e44dcc3857781'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.221883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc8e7d89ec243e4822e44dcc3857781'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.234521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc8e7d89ec243e4822e44dcc3857781'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.235619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc8e7d89ec243e4822e44dcc3857781'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.251267 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1046/2079] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:41 2022] DELETE /identity/v3/users/7cc8e7d89ec243e4822e44dcc3857781 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:41.256583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e96b-c3ae-446f-9799-c259994e96b2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:41.257059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e96b-c3ae-446f-9799-c259994e96b2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:41.257151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2e96b-c3ae-446f-9799-c259994e96b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:41.276874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dda2e96b-c3ae-446f-9799-c259994e96b2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:41.278207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dda2e96b-c3ae-446f-9799-c259994e96b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:41.344954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1034/2080] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:41.355058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:41.356484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:41.380203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:41.438575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467d2f5-c561-4f43-a908-946bb111de0d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:41.443087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa802-2fff-453c-9b31-b6c492b8fa8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:41.443703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa802-2fff-453c-9b31-b6c492b8fa8d service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:41.443987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa802-2fff-453c-9b31-b6c492b8fa8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:41.444268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa802-2fff-453c-9b31-b6c492b8fa8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:41.458458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafa802-2fff-453c-9b31-b6c492b8fa8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:41.459916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafa802-2fff-453c-9b31-b6c492b8fa8d service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:41.469897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1047/2081] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:41.775761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2e96b-c3ae-446f-9799-c259994e96b2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:41.777670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dda2e96b-c3ae-446f-9799-c259994e96b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:41.797754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2e96b-c3ae-446f-9799-c259994e96b2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:41.806178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:41.807145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:41.807539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:41.808010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] PATH_INFO: `/v3/projects/0d8de58eebfb47cea3822ead09d8ccf6` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:41.809421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8de58eebfb47cea3822ead09d8ccf6)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:41.811542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:41.868618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8de58eebfb47cea3822ead09d8ccf6', 'request_id': 'req-0e613b5e-770e-40a1-9b15-a373c7b03167'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.879817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8de58eebfb47cea3822ead09d8ccf6', 'request_id': 'req-0e613b5e-770e-40a1-9b15-a373c7b03167'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.893472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8de58eebfb47cea3822ead09d8ccf6', 'request_id': 'req-0e613b5e-770e-40a1-9b15-a373c7b03167'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.905449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:41.915421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] The token cache is being invalidate because project 0d8de58eebfb47cea3822ead09d8ccf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:41.915969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.916936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:41.922530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1035/2082] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:41 2022] DELETE /identity/v3/projects/0d8de58eebfb47cea3822ead09d8ccf6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:41.924783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa802-2fff-453c-9b31-b6c492b8fa8d service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:41.934879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa802-2fff-453c-9b31-b6c492b8fa8d service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:42.012620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:42.013464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:42.013974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:42.014344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] PATH_INFO: `/v3/users/dc2285e297f44393819d47c4f8473c46` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:42.017356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2285e297f44393819d47c4f8473c46)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:42.019776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:42.089460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2285e297f44393819d47c4f8473c46'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.099231 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2285e297f44393819d47c4f8473c46'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.100094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2285e297f44393819d47c4f8473c46'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.118545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2285e297f44393819d47c4f8473c46'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.120614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2285e297f44393819d47c4f8473c46'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.150150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1048/2083] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:41 2022] DELETE /identity/v3/users/dc2285e297f44393819d47c4f8473c46 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:42.465592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:42.467038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:42.483051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e613b5e-770e-40a1-9b15-a373c7b03167 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:42.557530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:42.558204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:42.558443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:42.558711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] PATH_INFO: `/v3/projects/2fbeca86c8de411c8eb47d24be7abb69` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:42.560551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fbeca86c8de411c8eb47d24be7abb69)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:42.562548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:42.630008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fbeca86c8de411c8eb47d24be7abb69', 'request_id': 'req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.637808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fbeca86c8de411c8eb47d24be7abb69', 'request_id': 'req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.645741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fbeca86c8de411c8eb47d24be7abb69', 'request_id': 'req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.657357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] The token cache is being invalidate because project 2fbeca86c8de411c8eb47d24be7abb69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:42.657558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.658819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.660495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1036/2084] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:42 2022] DELETE /identity/v3/projects/2fbeca86c8de411c8eb47d24be7abb69 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:42.664945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:42.666409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:42.680512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbad6e9-6674-4175-8758-fc9dfdb8aca4 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:42.728057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:42.749994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:42.750629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:42.750910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:42.751179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] PATH_INFO: `/v3/users/632c198b30ea4701922fdd0e0381cce9` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:42.752974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=632c198b30ea4701922fdd0e0381cce9)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:42.755096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:42.802941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '632c198b30ea4701922fdd0e0381cce9'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.809834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '632c198b30ea4701922fdd0e0381cce9'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.810521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '632c198b30ea4701922fdd0e0381cce9'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.823705 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '632c198b30ea4701922fdd0e0381cce9'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.824395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8d338750-d29d-4224-807e-3d643cb71e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '632c198b30ea4701922fdd0e0381cce9'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:42.839353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1049/2085] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:42 2022] DELETE /identity/v3/users/632c198b30ea4701922fdd0e0381cce9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:43.164496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:43.166163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:43.179684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd06782-2f7a-404e-a3c9-bd6f3f82b3c1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:43.262235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:43.263076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.263365 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.263638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] PATH_INFO: `/v3/projects/488ba340a2aa494e9b4a57febde6ac3d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.264866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=488ba340a2aa494e9b4a57febde6ac3d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:43.267551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:43.306352 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:43.321195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '488ba340a2aa494e9b4a57febde6ac3d', 'request_id': 'req-f162ba5c-a748-4c74-960b-4700f011a5fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:43.328801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '488ba340a2aa494e9b4a57febde6ac3d', 'request_id': 'req-f162ba5c-a748-4c74-960b-4700f011a5fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:43.336064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '488ba340a2aa494e9b4a57febde6ac3d', 'request_id': 'req-f162ba5c-a748-4c74-960b-4700f011a5fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:43.346883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] The token cache is being invalidate because project 488ba340a2aa494e9b4a57febde6ac3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:43.346883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:43.347565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f162ba5c-a748-4c74-960b-4700f011a5fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:43.349089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1037/2086] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:43 2022] DELETE /identity/v3/projects/488ba340a2aa494e9b4a57febde6ac3d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:43.355057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23644569-f96e-4663-a50b-b0d52b1b78e4 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.355531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23644569-f96e-4663-a50b-b0d52b1b78e4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.355647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23644569-f96e-4663-a50b-b0d52b1b78e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.375360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-23644569-f96e-4663-a50b-b0d52b1b78e4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:43.376593 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-23644569-f96e-4663-a50b-b0d52b1b78e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:43.441254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1050/2087] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:43.450920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfc748-17f9-4e91-b72f-597467ab6727 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.451231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfc748-17f9-4e91-b72f-597467ab6727 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.451511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfc748-17f9-4e91-b72f-597467ab6727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.472906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dacfc748-17f9-4e91-b72f-597467ab6727 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:43.474219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dacfc748-17f9-4e91-b72f-597467ab6727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:43.482377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1038/2088] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:43.487103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23644569-f96e-4663-a50b-b0d52b1b78e4 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:43.508817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23644569-f96e-4663-a50b-b0d52b1b78e4 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:43.513672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:43.514298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.514577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.514848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.515556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:19:43.515857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:43.517626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:43.535722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1051/2089] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:19:43.537015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfc748-17f9-4e91-b72f-597467ab6727 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:43.546983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfc748-17f9-4e91-b72f-597467ab6727 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:43.552845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c48d2-d01d-4778-93d7-cef53a501cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:43.553497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c48d2-d01d-4778-93d7-cef53a501cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.553772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c48d2-d01d-4778-93d7-cef53a501cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.554041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c48d2-d01d-4778-93d7-cef53a501cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.554690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c48d2-d01d-4778-93d7-cef53a501cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:43.556700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c48d2-d01d-4778-93d7-cef53a501cb2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:43.589458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1039/2090] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:43 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:43.593696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:43.605062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858d6c4-2c98-4cd9-ac43-a8c15fcbfe8d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:43.653427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:43.669439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de608d9e-0e0a-4612-9e15-1b413f9b2016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:43.670079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de608d9e-0e0a-4612-9e15-1b413f9b2016 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.670351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de608d9e-0e0a-4612-9e15-1b413f9b2016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.670627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de608d9e-0e0a-4612-9e15-1b413f9b2016 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.671392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de608d9e-0e0a-4612-9e15-1b413f9b2016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:43.673446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de608d9e-0e0a-4612-9e15-1b413f9b2016 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:43.696895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1052/2091] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:43 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:43.701273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c48d2-d01d-4778-93d7-cef53a501cb2 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:43.712908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c48d2-d01d-4778-93d7-cef53a501cb2 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:43.718092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0caf-246b-4569-b839-4fc32282aa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:43.718711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0caf-246b-4569-b839-4fc32282aa54 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.718996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0caf-246b-4569-b839-4fc32282aa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.719271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c0caf-246b-4569-b839-4fc32282aa54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.719934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c0caf-246b-4569-b839-4fc32282aa54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:43.721944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c0caf-246b-4569-b839-4fc32282aa54 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:43.732931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1040/2092] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:19:43.738605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de608d9e-0e0a-4612-9e15-1b413f9b2016 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:43.752854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de608d9e-0e0a-4612-9e15-1b413f9b2016 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:43.761160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0c8f1-9425-4afe-aaa2-1f9cf32bb0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:43.762003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0c8f1-9425-4afe-aaa2-1f9cf32bb0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.762355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0c8f1-9425-4afe-aaa2-1f9cf32bb0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.762691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0c8f1-9425-4afe-aaa2-1f9cf32bb0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.763565 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0c8f1-9425-4afe-aaa2-1f9cf32bb0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:43.765234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0c8f1-9425-4afe-aaa2-1f9cf32bb0b8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:43.777950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1053/2093] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:43.782402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0caf-246b-4569-b839-4fc32282aa54 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:43.794799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c0caf-246b-4569-b839-4fc32282aa54 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:43.802370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce197a-28dc-4777-a996-e4e60e81e32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:43.803072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce197a-28dc-4777-a996-e4e60e81e32d admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.803356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce197a-28dc-4777-a996-e4e60e81e32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.803632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce197a-28dc-4777-a996-e4e60e81e32d admin admin] PATH_INFO: `/v3/projects/5b8227e29f384fd08de4ec602ba6e974/users/4c348570d794494c9279b10ce244f7ad/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.823098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce197a-28dc-4777-a996-e4e60e81e32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8227e29f384fd08de4ec602ba6e974, user_id=4c348570d794494c9279b10ce244f7ad, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:43.826091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce197a-28dc-4777-a996-e4e60e81e32d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:43.844611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1041/2094] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:43 2022] PUT /identity/v3/projects/5b8227e29f384fd08de4ec602ba6e974/users/4c348570d794494c9279b10ce244f7ad/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:43.854015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf52ea-8813-43e9-aab6-aaf2ee73dd0a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:43.854301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf52ea-8813-43e9-aab6-aaf2ee73dd0a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:43.854570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf52ea-8813-43e9-aab6-aaf2ee73dd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:43.888180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f6bf52ea-8813-43e9-aab6-aaf2ee73dd0a None None] MFA Rules not processed for user `4c348570d794494c9279b10ce244f7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:43.889293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f6bf52ea-8813-43e9-aab6-aaf2ee73dd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:43.980879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1054/2095] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:43.986261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce197a-28dc-4777-a996-e4e60e81e32d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.052057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce197a-28dc-4777-a996-e4e60e81e32d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.056784 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece98dc-85aa-481e-bae7-f29cac1263fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.057382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece98dc-85aa-481e-bae7-f29cac1263fa admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.057659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece98dc-85aa-481e-bae7-f29cac1263fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.057924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece98dc-85aa-481e-bae7-f29cac1263fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.058563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece98dc-85aa-481e-bae7-f29cac1263fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.060482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece98dc-85aa-481e-bae7-f29cac1263fa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:44.096122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1042/2096] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:43 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:19:44.100920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf52ea-8813-43e9-aab6-aaf2ee73dd0a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.118062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf52ea-8813-43e9-aab6-aaf2ee73dd0a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.179726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591210-9582-4bdf-9c0e-4177217d58e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.180420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591210-9582-4bdf-9c0e-4177217d58e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.180682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591210-9582-4bdf-9c0e-4177217d58e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.180956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591210-9582-4bdf-9c0e-4177217d58e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.181598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da591210-9582-4bdf-9c0e-4177217d58e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.183580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da591210-9582-4bdf-9c0e-4177217d58e1 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:44.207162 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1055/2097] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:44 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:44.211610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece98dc-85aa-481e-bae7-f29cac1263fa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.223987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece98dc-85aa-481e-bae7-f29cac1263fa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.229623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729e879-697b-4fcd-a075-cac2e2d229b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.230361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729e879-697b-4fcd-a075-cac2e2d229b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.230639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729e879-697b-4fcd-a075-cac2e2d229b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.230973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729e879-697b-4fcd-a075-cac2e2d229b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.231593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729e879-697b-4fcd-a075-cac2e2d229b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.233250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729e879-697b-4fcd-a075-cac2e2d229b7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:44.246686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1043/2098] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:19:44.250867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591210-9582-4bdf-9c0e-4177217d58e1 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.263068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591210-9582-4bdf-9c0e-4177217d58e1 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.268928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb22b-d57f-4015-a840-b913c5aca7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.269643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb22b-d57f-4015-a840-b913c5aca7ff admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.269852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb22b-d57f-4015-a840-b913c5aca7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.270131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb22b-d57f-4015-a840-b913c5aca7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.270784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb22b-d57f-4015-a840-b913c5aca7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.272454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb22b-d57f-4015-a840-b913c5aca7ff admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:44.284014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1056/2099] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:44.290559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729e879-697b-4fcd-a075-cac2e2d229b7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.309253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729e879-697b-4fcd-a075-cac2e2d229b7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.317511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c7696-c824-49af-b835-f5204e9545e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.318928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c7696-c824-49af-b835-f5204e9545e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.319401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c7696-c824-49af-b835-f5204e9545e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.319868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c7696-c824-49af-b835-f5204e9545e6 admin admin] PATH_INFO: `/v3/projects/b8d87fb3572246099a1301f24211a804/users/bfdefa46a0fd431db1c99abdd27c7874/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.343592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c7696-c824-49af-b835-f5204e9545e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d87fb3572246099a1301f24211a804, user_id=bfdefa46a0fd431db1c99abdd27c7874, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.347720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c7696-c824-49af-b835-f5204e9545e6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:44.373566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1044/2100] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:44 2022] PUT /identity/v3/projects/b8d87fb3572246099a1301f24211a804/users/bfdefa46a0fd431db1c99abdd27c7874/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:44.385266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1db5-dd00-465b-a6f7-623ac3cc30c3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.386147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1db5-dd00-465b-a6f7-623ac3cc30c3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.386666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1db5-dd00-465b-a6f7-623ac3cc30c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.455979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-12bc1db5-dd00-465b-a6f7-623ac3cc30c3 None None] MFA Rules not processed for user `bfdefa46a0fd431db1c99abdd27c7874`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:44.462991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-12bc1db5-dd00-465b-a6f7-623ac3cc30c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:44.524077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:44.588363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1057/2101] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:44.593188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c7696-c824-49af-b835-f5204e9545e6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.604947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:44.690498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c7696-c824-49af-b835-f5204e9545e6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.696433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deee7f-ebdf-41a8-a4ad-6bf1f6b3f280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.697103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deee7f-ebdf-41a8-a4ad-6bf1f6b3f280 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.697381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deee7f-ebdf-41a8-a4ad-6bf1f6b3f280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.697667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deee7f-ebdf-41a8-a4ad-6bf1f6b3f280 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.698349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deee7f-ebdf-41a8-a4ad-6bf1f6b3f280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.700344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20deee7f-ebdf-41a8-a4ad-6bf1f6b3f280 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:44.741178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1045/2102] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:44 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:44.742645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc1db5-dd00-465b-a6f7-623ac3cc30c3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.763608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc1db5-dd00-465b-a6f7-623ac3cc30c3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.849361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32746937-61e2-4d06-8d77-881522199917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.850214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32746937-61e2-4d06-8d77-881522199917 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.850622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32746937-61e2-4d06-8d77-881522199917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.851175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32746937-61e2-4d06-8d77-881522199917 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.852166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32746937-61e2-4d06-8d77-881522199917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.854323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32746937-61e2-4d06-8d77-881522199917 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:44.883925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1058/2103] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:44 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:44.887786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deee7f-ebdf-41a8-a4ad-6bf1f6b3f280 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.901357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deee7f-ebdf-41a8-a4ad-6bf1f6b3f280 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.906614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87917228-d1e4-4462-a83c-450f6933d03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.907230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87917228-d1e4-4462-a83c-450f6933d03f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.907532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87917228-d1e4-4462-a83c-450f6933d03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.907791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87917228-d1e4-4462-a83c-450f6933d03f admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.909065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87917228-d1e4-4462-a83c-450f6933d03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.910695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87917228-d1e4-4462-a83c-450f6933d03f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:44.926646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1046/2104] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:44.930836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32746937-61e2-4d06-8d77-881522199917 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:44.943491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32746937-61e2-4d06-8d77-881522199917 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:44.951453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631135d-ea46-4b33-b932-566f35d4fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:44.952300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631135d-ea46-4b33-b932-566f35d4fbdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:44.952812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631135d-ea46-4b33-b932-566f35d4fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:44.953173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631135d-ea46-4b33-b932-566f35d4fbdb admin admin] PATH_INFO: `/v3/projects/042ca14697b04dadbe2a0dd591b285e7/users/27c6d0ce054d45d3bc63c86603d4494a/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:44.978668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631135d-ea46-4b33-b932-566f35d4fbdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=042ca14697b04dadbe2a0dd591b285e7, user_id=27c6d0ce054d45d3bc63c86603d4494a, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:44.981696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631135d-ea46-4b33-b932-566f35d4fbdb admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:45.004915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1059/2105] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:44 2022] PUT /identity/v3/projects/042ca14697b04dadbe2a0dd591b285e7/users/27c6d0ce054d45d3bc63c86603d4494a/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:45.006338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87917228-d1e4-4462-a83c-450f6933d03f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:45.017095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87917228-d1e4-4462-a83c-450f6933d03f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:45.100643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff7ba3-6681-4e75-9a4d-dfcec7ad27c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:45.101508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff7ba3-6681-4e75-9a4d-dfcec7ad27c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:45.101855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff7ba3-6681-4e75-9a4d-dfcec7ad27c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:45.102287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff7ba3-6681-4e75-9a4d-dfcec7ad27c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:45.102943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff7ba3-6681-4e75-9a4d-dfcec7ad27c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:45.104608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff7ba3-6681-4e75-9a4d-dfcec7ad27c0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:45.116963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1047/2106] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:45.122180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631135d-ea46-4b33-b932-566f35d4fbdb admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:45.135181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631135d-ea46-4b33-b932-566f35d4fbdb admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:45.143237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:45.143984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:45.144337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:45.144720 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] PATH_INFO: `/v3/projects/042ca14697b04dadbe2a0dd591b285e7/users/27c6d0ce054d45d3bc63c86603d4494a/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:45.148488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=042ca14697b04dadbe2a0dd591b285e7, user_id=27c6d0ce054d45d3bc63c86603d4494a, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:45.151203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:45.163139 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1060/2107] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:45 2022] PUT /identity/v3/projects/042ca14697b04dadbe2a0dd591b285e7/users/27c6d0ce054d45d3bc63c86603d4494a/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:45.174742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a589f-f86f-4aa3-b328-afb9df9de72f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:45.175099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a589f-f86f-4aa3-b328-afb9df9de72f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:45.175436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a589f-f86f-4aa3-b328-afb9df9de72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:45.208380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e82a589f-f86f-4aa3-b328-afb9df9de72f None None] MFA Rules not processed for user `27c6d0ce054d45d3bc63c86603d4494a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:45.209835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e82a589f-f86f-4aa3-b328-afb9df9de72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:45.281705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1048/2108] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:45.294168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:45.295870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:45.359089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34257f43-b39b-4424-8619-01eaeba86251 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:45.364591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:45.365223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:45.365793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:45.366005 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:45.378428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:45.379981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:45.393881 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1061/2109] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:45.446914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a589f-f86f-4aa3-b328-afb9df9de72f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:45.452102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:45.459957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a589f-f86f-4aa3-b328-afb9df9de72f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:45.463771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a55b2e-d125-4234-aad8-b69170f3c1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:45.464316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a55b2e-d125-4234-aad8-b69170f3c1ad service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:45.464644 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a55b2e-d125-4234-aad8-b69170f3c1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:45.464917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a55b2e-d125-4234-aad8-b69170f3c1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:45.548793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a55b2e-d125-4234-aad8-b69170f3c1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:45.552114 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a55b2e-d125-4234-aad8-b69170f3c1ad service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:45.574508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1049/2110] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:53.177076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:53.179914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:53.207979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:53.252400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab47132-2549-4549-9fde-46c928e4fa44 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:53.258081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730401a-34c3-4f54-ad4b-12cf5995f48e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:53.258081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730401a-34c3-4f54-ad4b-12cf5995f48e service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:53.258081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730401a-34c3-4f54-ad4b-12cf5995f48e service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:53.258081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730401a-34c3-4f54-ad4b-12cf5995f48e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:53.260116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9730401a-34c3-4f54-ad4b-12cf5995f48e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:53.334983 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730401a-34c3-4f54-ad4b-12cf5995f48e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:53.336455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730401a-34c3-4f54-ad4b-12cf5995f48e service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:53.350521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1062/2111] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:53.983276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a55b2e-d125-4234-aad8-b69170f3c1ad service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:54.054105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a55b2e-d125-4234-aad8-b69170f3c1ad service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:54.058691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:54.059313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:54.059587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:54.059857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] PATH_INFO: `/v3/users/4c348570d794494c9279b10ce244f7ad` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:54.061007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c348570d794494c9279b10ce244f7ad)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:54.063054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:54.115713 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c348570d794494c9279b10ce244f7ad'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:54.130215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c348570d794494c9279b10ce244f7ad'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:54.130966 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c348570d794494c9279b10ce244f7ad'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:54.131564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c348570d794494c9279b10ce244f7ad'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:54.151481 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c348570d794494c9279b10ce244f7ad'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:54.171028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1050/2112] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:53 2022] DELETE /identity/v3/users/4c348570d794494c9279b10ce244f7ad => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:54.175976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab431064-6bfb-4ed0-8328-8f042af96fbc None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:54.176273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab431064-6bfb-4ed0-8328-8f042af96fbc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:54.176552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab431064-6bfb-4ed0-8328-8f042af96fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:54.201554 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-ab431064-6bfb-4ed0-8328-8f042af96fbc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:54.202778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ab431064-6bfb-4ed0-8328-8f042af96fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:54.299688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1063/2113] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:54.313406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:54.316076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:54.342485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:54.401260 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45fa9c-ea1e-4509-a259-136fe6a9ec63 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:54.405483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7da14-038a-4dd3-8f75-a22102f79a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:54.406117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7da14-038a-4dd3-8f75-a22102f79a61 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:54.406395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7da14-038a-4dd3-8f75-a22102f79a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:54.406666 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7da14-038a-4dd3-8f75-a22102f79a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:54.419198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7da14-038a-4dd3-8f75-a22102f79a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:54.420924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7da14-038a-4dd3-8f75-a22102f79a61 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:54.433413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1051/2114] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:55.584155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab431064-6bfb-4ed0-8328-8f042af96fbc None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:55.585807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ab431064-6bfb-4ed0-8328-8f042af96fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:55.600678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab431064-6bfb-4ed0-8328-8f042af96fbc None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:55.605326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:55.605988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.606246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.606522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] PATH_INFO: `/v3/projects/5b8227e29f384fd08de4ec602ba6e974` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.607700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8227e29f384fd08de4ec602ba6e974)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:55.609698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:55.663493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8227e29f384fd08de4ec602ba6e974', 'request_id': 'req-81e87eb2-c090-40d7-b8c5-611dbfde9640'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.670083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8227e29f384fd08de4ec602ba6e974', 'request_id': 'req-81e87eb2-c090-40d7-b8c5-611dbfde9640'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.677807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8227e29f384fd08de4ec602ba6e974', 'request_id': 'req-81e87eb2-c090-40d7-b8c5-611dbfde9640'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.679563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:55.697654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] The token cache is being invalidate because project 5b8227e29f384fd08de4ec602ba6e974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:55.697947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.698604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.700163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1064/2115] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:55 2022] DELETE /identity/v3/projects/5b8227e29f384fd08de4ec602ba6e974 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:55.704186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7da14-038a-4dd3-8f75-a22102f79a61 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:55.715010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7da14-038a-4dd3-8f75-a22102f79a61 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:55.790513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:55.791149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.791430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.791703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] PATH_INFO: `/v3/users/bfdefa46a0fd431db1c99abdd27c7874` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.792869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfdefa46a0fd431db1c99abdd27c7874)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:55.794821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:55.848953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfdefa46a0fd431db1c99abdd27c7874'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.867529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfdefa46a0fd431db1c99abdd27c7874'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.868304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfdefa46a0fd431db1c99abdd27c7874'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.868961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfdefa46a0fd431db1c99abdd27c7874'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.878067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfdefa46a0fd431db1c99abdd27c7874'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:55.891390 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1052/2116] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:55 2022] DELETE /identity/v3/users/bfdefa46a0fd431db1c99abdd27c7874 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:56.251432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:56.253646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:56.267287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e87eb2-c090-40d7-b8c5-611dbfde9640 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:56.330908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:56.331639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:56.331935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:56.332184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] PATH_INFO: `/v3/projects/b8d87fb3572246099a1301f24211a804` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:56.333377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d87fb3572246099a1301f24211a804)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:56.335310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:56.387601 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d87fb3572246099a1301f24211a804', 'request_id': 'req-3da96340-dd69-4948-8edf-e30db5726b3c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.395130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d87fb3572246099a1301f24211a804', 'request_id': 'req-3da96340-dd69-4948-8edf-e30db5726b3c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.402006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d87fb3572246099a1301f24211a804', 'request_id': 'req-3da96340-dd69-4948-8edf-e30db5726b3c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.419250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] The token cache is being invalidate because project b8d87fb3572246099a1301f24211a804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:56.419760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.420329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.422142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1065/2117] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:56 2022] DELETE /identity/v3/projects/b8d87fb3572246099a1301f24211a804 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:56.426913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:56.428914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:56.444211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c00b41-7535-4c4e-9181-179969dcfcc5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:56.493990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:56.514928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:56.515573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:56.515865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:56.516156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] PATH_INFO: `/v3/users/27c6d0ce054d45d3bc63c86603d4494a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:56.517321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c6d0ce054d45d3bc63c86603d4494a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:56.519638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:56.573142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c6d0ce054d45d3bc63c86603d4494a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.588138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c6d0ce054d45d3bc63c86603d4494a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.589216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c6d0ce054d45d3bc63c86603d4494a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.589758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c6d0ce054d45d3bc63c86603d4494a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.599011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7c936115-07ed-48cd-b6c9-b77782affed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c6d0ce054d45d3bc63c86603d4494a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:56.618283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1053/2118] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:56 2022] DELETE /identity/v3/users/27c6d0ce054d45d3bc63c86603d4494a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:56.930753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:56.933932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:56.951020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da96340-dd69-4948-8edf-e30db5726b3c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.010890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.011516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.011792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.012062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] PATH_INFO: `/v3/projects/042ca14697b04dadbe2a0dd591b285e7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.013212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=042ca14697b04dadbe2a0dd591b285e7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.015119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.040490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:57.066591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042ca14697b04dadbe2a0dd591b285e7', 'request_id': 'req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:57.072795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042ca14697b04dadbe2a0dd591b285e7', 'request_id': 'req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:57.078291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042ca14697b04dadbe2a0dd591b285e7', 'request_id': 'req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:57.091628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] The token cache is being invalidate because project 042ca14697b04dadbe2a0dd591b285e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:57.091913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:57.092526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-92128a8a-6eb5-4589-9628-a3d6c57ee0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:57.094122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1066/2119] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:56 2022] DELETE /identity/v3/projects/042ca14697b04dadbe2a0dd591b285e7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:57.101551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8243-9ea9-424f-92aa-f7d33436882b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.102052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8243-9ea9-424f-92aa-f7d33436882b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.102164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8243-9ea9-424f-92aa-f7d33436882b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.124793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-54fd8243-9ea9-424f-92aa-f7d33436882b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:57.126054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-54fd8243-9ea9-424f-92aa-f7d33436882b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:57.191938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1054/2120] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:57.201245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf5764-936a-4217-922b-80ad36dcc526 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.201534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf5764-936a-4217-922b-80ad36dcc526 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.201838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf5764-936a-4217-922b-80ad36dcc526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.223550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-bfaf5764-936a-4217-922b-80ad36dcc526 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:57.224787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-bfaf5764-936a-4217-922b-80ad36dcc526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:57.232534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1067/2121] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:57.237580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd8243-9ea9-424f-92aa-f7d33436882b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.247299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd8243-9ea9-424f-92aa-f7d33436882b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.251893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.253123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.253404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.253677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.254323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:19:57.254626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.256232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.273090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1055/2122] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:19:57.274198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf5764-936a-4217-922b-80ad36dcc526 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.284711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf5764-936a-4217-922b-80ad36dcc526 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.289766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85ae7d-1a9d-4c80-b913-4181564d2c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.290372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85ae7d-1a9d-4c80-b913-4181564d2c90 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.290651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85ae7d-1a9d-4c80-b913-4181564d2c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.290922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85ae7d-1a9d-4c80-b913-4181564d2c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.291559 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85ae7d-1a9d-4c80-b913-4181564d2c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.293620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85ae7d-1a9d-4c80-b913-4181564d2c90 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.324505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1068/2123] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:57 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:57.326404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.339797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d393832-31cc-4dd2-80d3-d20af94ac8e3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.389855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:57.405313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7411adf-ba18-43b6-af36-5f2798eaf991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.405944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7411adf-ba18-43b6-af36-5f2798eaf991 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.406426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7411adf-ba18-43b6-af36-5f2798eaf991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.406764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7411adf-ba18-43b6-af36-5f2798eaf991 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.407799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7411adf-ba18-43b6-af36-5f2798eaf991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.410371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7411adf-ba18-43b6-af36-5f2798eaf991 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.437383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1056/2124] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:57 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:19:57.440582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85ae7d-1a9d-4c80-b913-4181564d2c90 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.451287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85ae7d-1a9d-4c80-b913-4181564d2c90 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.456342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d9282-d809-49c9-9cac-8f896291ecb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.456963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d9282-d809-49c9-9cac-8f896291ecb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.457241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d9282-d809-49c9-9cac-8f896291ecb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.457511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d9282-d809-49c9-9cac-8f896291ecb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.458148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d9282-d809-49c9-9cac-8f896291ecb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.459744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d9282-d809-49c9-9cac-8f896291ecb6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.469756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1069/2125] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:19:57.474573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7411adf-ba18-43b6-af36-5f2798eaf991 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.488010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7411adf-ba18-43b6-af36-5f2798eaf991 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.494186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cff8cb-29f3-40cb-a449-8bb27e759691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.494873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff8cb-29f3-40cb-a449-8bb27e759691 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.495224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff8cb-29f3-40cb-a449-8bb27e759691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.495566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cff8cb-29f3-40cb-a449-8bb27e759691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.496290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cff8cb-29f3-40cb-a449-8bb27e759691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.498036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cff8cb-29f3-40cb-a449-8bb27e759691 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.510585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1057/2126] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:57.514832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d9282-d809-49c9-9cac-8f896291ecb6 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.534741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d9282-d809-49c9-9cac-8f896291ecb6 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.539726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806db2-180a-4867-a78a-bd7534bcb4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.540379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806db2-180a-4867-a78a-bd7534bcb4e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.540703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806db2-180a-4867-a78a-bd7534bcb4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.540979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806db2-180a-4867-a78a-bd7534bcb4e4 admin admin] PATH_INFO: `/v3/projects/81dd8b849dfa431780279ed9b8001d2b/users/4957265d46cd4b4587661df4b050af8a/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.555754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f806db2-180a-4867-a78a-bd7534bcb4e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81dd8b849dfa431780279ed9b8001d2b, user_id=4957265d46cd4b4587661df4b050af8a, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.558487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f806db2-180a-4867-a78a-bd7534bcb4e4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.572877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1070/2127] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:57 2022] PUT /identity/v3/projects/81dd8b849dfa431780279ed9b8001d2b/users/4957265d46cd4b4587661df4b050af8a/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:57.585755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0644c-932e-44d0-ad39-7c14d7c1b178 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.586041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0644c-932e-44d0-ad39-7c14d7c1b178 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.586311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0644c-932e-44d0-ad39-7c14d7c1b178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.613805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a6e0644c-932e-44d0-ad39-7c14d7c1b178 None None] MFA Rules not processed for user `4957265d46cd4b4587661df4b050af8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:57.615025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a6e0644c-932e-44d0-ad39-7c14d7c1b178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:57.681989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1058/2128] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:57.686655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806db2-180a-4867-a78a-bd7534bcb4e4 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.753095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806db2-180a-4867-a78a-bd7534bcb4e4 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.757811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb3ed2-d965-49e5-b5c4-a3a1b7ca4d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.758452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb3ed2-d965-49e5-b5c4-a3a1b7ca4d83 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.758731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb3ed2-d965-49e5-b5c4-a3a1b7ca4d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.759111 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb3ed2-d965-49e5-b5c4-a3a1b7ca4d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.759771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb3ed2-d965-49e5-b5c4-a3a1b7ca4d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.761726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb3ed2-d965-49e5-b5c4-a3a1b7ca4d83 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.793235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1071/2129] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:19:57 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:57.797409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0644c-932e-44d0-ad39-7c14d7c1b178 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.808802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0644c-932e-44d0-ad39-7c14d7c1b178 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.866243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2584b8c-0643-4947-a9ea-0cca7604d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.866857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2584b8c-0643-4947-a9ea-0cca7604d0ca admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.867135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2584b8c-0643-4947-a9ea-0cca7604d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.867409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2584b8c-0643-4947-a9ea-0cca7604d0ca admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.868112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2584b8c-0643-4947-a9ea-0cca7604d0ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.870111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2584b8c-0643-4947-a9ea-0cca7604d0ca admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.892407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1059/2130] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:19:57 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:19:57.896503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb3ed2-d965-49e5-b5c4-a3a1b7ca4d83 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.907010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb3ed2-d965-49e5-b5c4-a3a1b7ca4d83 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.912596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f101d2-c364-43b9-a63a-af964cec1b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.913458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f101d2-c364-43b9-a63a-af964cec1b90 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.913741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f101d2-c364-43b9-a63a-af964cec1b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.914015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f101d2-c364-43b9-a63a-af964cec1b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.914661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f101d2-c364-43b9-a63a-af964cec1b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.916276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f101d2-c364-43b9-a63a-af964cec1b90 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.928340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1072/2131] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:57.932697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2584b8c-0643-4947-a9ea-0cca7604d0ca admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.943106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2584b8c-0643-4947-a9ea-0cca7604d0ca admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:57.947950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a11ca-2295-455d-a145-a445b15656df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:57.948635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a11ca-2295-455d-a145-a445b15656df admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:57.948918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a11ca-2295-455d-a145-a445b15656df admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:57.949195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a11ca-2295-455d-a145-a445b15656df admin admin] PATH_INFO: `/v3/projects/fd194fd08058497382d75da7df77533f/users/a0c81e6bca14463c86a131b56d64f8cc/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:57.964602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a11ca-2295-455d-a145-a445b15656df admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd194fd08058497382d75da7df77533f, user_id=a0c81e6bca14463c86a131b56d64f8cc, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:57.967258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a11ca-2295-455d-a145-a445b15656df admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:57.981353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1060/2132] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:57 2022] PUT /identity/v3/projects/fd194fd08058497382d75da7df77533f/users/a0c81e6bca14463c86a131b56d64f8cc/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:57.985630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f101d2-c364-43b9-a63a-af964cec1b90 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:57.996642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f101d2-c364-43b9-a63a-af964cec1b90 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:58.015478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:58.057021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fdbd7-4858-4cd2-8073-cdc593cea365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:58.057618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fdbd7-4858-4cd2-8073-cdc593cea365 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:58.057902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fdbd7-4858-4cd2-8073-cdc593cea365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:58.058175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fdbd7-4858-4cd2-8073-cdc593cea365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:58.058822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fdbd7-4858-4cd2-8073-cdc593cea365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:58.060436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fdbd7-4858-4cd2-8073-cdc593cea365 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:58.071499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1073/2133] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:19:58.075895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a11ca-2295-455d-a145-a445b15656df admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:58.086542 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a11ca-2295-455d-a145-a445b15656df admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:58.091318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:58.091989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:58.092274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:58.092547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] PATH_INFO: `/v3/projects/fd194fd08058497382d75da7df77533f/users/a0c81e6bca14463c86a131b56d64f8cc/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:58.094545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd194fd08058497382d75da7df77533f, user_id=a0c81e6bca14463c86a131b56d64f8cc, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:58.097476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:58.108005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1061/2134] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:19:58 2022] PUT /identity/v3/projects/fd194fd08058497382d75da7df77533f/users/a0c81e6bca14463c86a131b56d64f8cc/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:58.118989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb37a76-786c-4251-a0dd-67ea2409f228 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:58.119359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb37a76-786c-4251-a0dd-67ea2409f228 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:58.119639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb37a76-786c-4251-a0dd-67ea2409f228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:58.151402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-3fb37a76-786c-4251-a0dd-67ea2409f228 None None] MFA Rules not processed for user `a0c81e6bca14463c86a131b56d64f8cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:58.152773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3fb37a76-786c-4251-a0dd-67ea2409f228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:58.223555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1074/2135] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:19:58.235523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:58.238269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:58.249514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:58.300537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba6994-3409-4711-b6d6-32a10732ed4f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:58.304186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc77c68-f804-41dc-b685-4ebfaf8f1e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:58.304820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc77c68-f804-41dc-b685-4ebfaf8f1e00 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:58.305099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc77c68-f804-41dc-b685-4ebfaf8f1e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:58.305368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc77c68-f804-41dc-b685-4ebfaf8f1e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:58.314294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc77c68-f804-41dc-b685-4ebfaf8f1e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:58.316389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc77c68-f804-41dc-b685-4ebfaf8f1e00 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:58.325823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1062/2136] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:58.966081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb37a76-786c-4251-a0dd-67ea2409f228 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:59.030402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb37a76-786c-4251-a0dd-67ea2409f228 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:59.035389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:59.036028 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:59.036308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:59.036595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] PATH_INFO: `/v3/users/4957265d46cd4b4587661df4b050af8a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:59.038042 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4957265d46cd4b4587661df4b050af8a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:59.039993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:59.108677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:59.110673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4957265d46cd4b4587661df4b050af8a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.128460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4957265d46cd4b4587661df4b050af8a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.129311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4957265d46cd4b4587661df4b050af8a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.149263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4957265d46cd4b4587661df4b050af8a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.150038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4957265d46cd4b4587661df4b050af8a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.165065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1075/2137] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:58 2022] DELETE /identity/v3/users/4957265d46cd4b4587661df4b050af8a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:59.169902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9906e-7836-4ec0-a5a4-dc48644e3434 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:59.170433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9906e-7836-4ec0-a5a4-dc48644e3434 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:59.171266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9906e-7836-4ec0-a5a4-dc48644e3434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:59.191978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-53e9906e-7836-4ec0-a5a4-dc48644e3434 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:59.193517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-53e9906e-7836-4ec0-a5a4-dc48644e3434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:59.258594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1063/2138] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:59.269489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:59.271002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:59.338343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd24d6-751a-4bf0-8b93-e5341480bed6 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:59.342819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a1cb6-2801-4a47-a091-6559c011d93d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:59.343430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1cb6-2801-4a47-a091-6559c011d93d service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:59.343706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1cb6-2801-4a47-a091-6559c011d93d service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:59.344171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1cb6-2801-4a47-a091-6559c011d93d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:59.363590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a1cb6-2801-4a47-a091-6559c011d93d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:59.365123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a1cb6-2801-4a47-a091-6559c011d93d service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:59.375247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1076/2139] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:59.658397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9906e-7836-4ec0-a5a4-dc48644e3434 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:59.661143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-53e9906e-7836-4ec0-a5a4-dc48644e3434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:59.676337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9906e-7836-4ec0-a5a4-dc48644e3434 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:59.681596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:59.682348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:59.682751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:59.683452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] PATH_INFO: `/v3/projects/81dd8b849dfa431780279ed9b8001d2b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:59.684347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=81dd8b849dfa431780279ed9b8001d2b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:59.686807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:59.742313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81dd8b849dfa431780279ed9b8001d2b', 'request_id': 'req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.750332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81dd8b849dfa431780279ed9b8001d2b', 'request_id': 'req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.758045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81dd8b849dfa431780279ed9b8001d2b', 'request_id': 'req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.769327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] The token cache is being invalidate because project 81dd8b849dfa431780279ed9b8001d2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:59.769766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.770434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.771940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1064/2140] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:59 2022] DELETE /identity/v3/projects/81dd8b849dfa431780279ed9b8001d2b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:59.776016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a1cb6-2801-4a47-a091-6559c011d93d service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:59.786084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a1cb6-2801-4a47-a091-6559c011d93d service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:59.844979 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:59.846225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:59.846506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:59.846784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] PATH_INFO: `/v3/users/a0c81e6bca14463c86a131b56d64f8cc` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:59.848071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0c81e6bca14463c86a131b56d64f8cc)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:59.850298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:59.903969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0c81e6bca14463c86a131b56d64f8cc'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.914945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0c81e6bca14463c86a131b56d64f8cc'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.915670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0c81e6bca14463c86a131b56d64f8cc'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.923544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:59.930257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0c81e6bca14463c86a131b56d64f8cc'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.931577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0bc6161-0b81-47fd-bef5-799328484074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0c81e6bca14463c86a131b56d64f8cc'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:59.949693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1077/2141] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:59 2022] DELETE /identity/v3/users/a0c81e6bca14463c86a131b56d64f8cc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:00.227007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:00.228475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:00.242717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5003e-5dbc-4c4a-a18c-068b6bf41f9e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:00.279558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:00.308857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:00.309540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.309972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.310338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] PATH_INFO: `/v3/projects/fd194fd08058497382d75da7df77533f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.313933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd194fd08058497382d75da7df77533f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:00.316655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:00.367488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd194fd08058497382d75da7df77533f', 'request_id': 'req-96155acd-53b6-4ed1-8d82-928c5f920598'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:00.374934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd194fd08058497382d75da7df77533f', 'request_id': 'req-96155acd-53b6-4ed1-8d82-928c5f920598'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:00.382015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd194fd08058497382d75da7df77533f', 'request_id': 'req-96155acd-53b6-4ed1-8d82-928c5f920598'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:00.392626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] The token cache is being invalidate because project fd194fd08058497382d75da7df77533f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:20:00.393103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:00.394171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96155acd-53b6-4ed1-8d82-928c5f920598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:00.395788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1065/2142] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:20:00 2022] DELETE /identity/v3/projects/fd194fd08058497382d75da7df77533f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:00.401183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739347ce-b843-4c8b-95e0-d3d9170870a8 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.401674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739347ce-b843-4c8b-95e0-d3d9170870a8 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.401771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739347ce-b843-4c8b-95e0-d3d9170870a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.421380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-739347ce-b843-4c8b-95e0-d3d9170870a8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:00.423101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-739347ce-b843-4c8b-95e0-d3d9170870a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:00.482036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1078/2143] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:00.490727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99881cca-62ff-44e1-9465-1a58b2dc6208 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.491018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99881cca-62ff-44e1-9465-1a58b2dc6208 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.491293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99881cca-62ff-44e1-9465-1a58b2dc6208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.511834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-99881cca-62ff-44e1-9465-1a58b2dc6208 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:00.513111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-99881cca-62ff-44e1-9465-1a58b2dc6208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:00.521377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1066/2144] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:00.525742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739347ce-b843-4c8b-95e0-d3d9170870a8 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:00.535410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739347ce-b843-4c8b-95e0-d3d9170870a8 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:00.540198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:00.540816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.541093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.541359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.542004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:20:00.542302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:00.543923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:00.555761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1079/2145] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:20:00.560197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99881cca-62ff-44e1-9465-1a58b2dc6208 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:00.570176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99881cca-62ff-44e1-9465-1a58b2dc6208 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:00.575025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7994abc-a358-4549-96d1-4ca23f068c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:00.575626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7994abc-a358-4549-96d1-4ca23f068c2e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.575908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7994abc-a358-4549-96d1-4ca23f068c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.576178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7994abc-a358-4549-96d1-4ca23f068c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.576860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7994abc-a358-4549-96d1-4ca23f068c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:00.578769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7994abc-a358-4549-96d1-4ca23f068c2e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:00.606877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1067/2146] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:20:00 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:00.611103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:00.622469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd113a-490e-4ad9-8700-3f8d798a1575 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:00.679746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae9fba-821d-4964-bc84-16ba50ea2544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:00.680354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae9fba-821d-4964-bc84-16ba50ea2544 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.680674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae9fba-821d-4964-bc84-16ba50ea2544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.680955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae9fba-821d-4964-bc84-16ba50ea2544 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.681595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae9fba-821d-4964-bc84-16ba50ea2544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:00.683531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae9fba-821d-4964-bc84-16ba50ea2544 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:00.706036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1080/2147] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:20:00 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:00.710164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7994abc-a358-4549-96d1-4ca23f068c2e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:00.722526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7994abc-a358-4549-96d1-4ca23f068c2e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:00.727961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8078286-7686-40aa-ac9d-e01fdf1544c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:00.728684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078286-7686-40aa-ac9d-e01fdf1544c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.729032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078286-7686-40aa-ac9d-e01fdf1544c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.729310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078286-7686-40aa-ac9d-e01fdf1544c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.729959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8078286-7686-40aa-ac9d-e01fdf1544c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:00.731563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8078286-7686-40aa-ac9d-e01fdf1544c3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:00.743116 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1068/2148] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:00.747082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae9fba-821d-4964-bc84-16ba50ea2544 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:00.759011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae9fba-821d-4964-bc84-16ba50ea2544 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:00.763714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a81d3-30db-47ac-9163-e9d063cb1e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:00.764317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a81d3-30db-47ac-9163-e9d063cb1e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.764611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a81d3-30db-47ac-9163-e9d063cb1e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.764883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a81d3-30db-47ac-9163-e9d063cb1e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.765529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a81d3-30db-47ac-9163-e9d063cb1e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:00.767144 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a81d3-30db-47ac-9163-e9d063cb1e9e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:00.779902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1081/2149] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:00.784664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8078286-7686-40aa-ac9d-e01fdf1544c3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:00.799517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8078286-7686-40aa-ac9d-e01fdf1544c3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:00.804816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59570d8e-41b6-4960-ad91-2dcac3d51476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:00.805847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59570d8e-41b6-4960-ad91-2dcac3d51476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.806226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59570d8e-41b6-4960-ad91-2dcac3d51476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.806709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59570d8e-41b6-4960-ad91-2dcac3d51476 admin admin] PATH_INFO: `/v3/projects/0d33a0eab9ea4056ad9a29a989d7ad34/users/7534951b063c4adcbf14b08d7452b824/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.824035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59570d8e-41b6-4960-ad91-2dcac3d51476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d33a0eab9ea4056ad9a29a989d7ad34, user_id=7534951b063c4adcbf14b08d7452b824, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:00.827193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59570d8e-41b6-4960-ad91-2dcac3d51476 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:00.849660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1069/2150] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:00 2022] PUT /identity/v3/projects/0d33a0eab9ea4056ad9a29a989d7ad34/users/7534951b063c4adcbf14b08d7452b824/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:20:00.860839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ab302-c7c8-44fb-916d-478a5984c8d1 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:00.861261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ab302-c7c8-44fb-916d-478a5984c8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:00.862220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ab302-c7c8-44fb-916d-478a5984c8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:00.913416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-bd6ab302-c7c8-44fb-916d-478a5984c8d1 None None] MFA Rules not processed for user `7534951b063c4adcbf14b08d7452b824`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:00.915936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-bd6ab302-c7c8-44fb-916d-478a5984c8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:00.981366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:01.018206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1082/2151] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:01.025560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59570d8e-41b6-4960-ad91-2dcac3d51476 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:01.106520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59570d8e-41b6-4960-ad91-2dcac3d51476 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:01.111546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64031c-e0be-4b94-80bb-d3767a8fb333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:01.112162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64031c-e0be-4b94-80bb-d3767a8fb333 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:01.112445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64031c-e0be-4b94-80bb-d3767a8fb333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:01.112753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64031c-e0be-4b94-80bb-d3767a8fb333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:01.113400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64031c-e0be-4b94-80bb-d3767a8fb333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:01.115472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64031c-e0be-4b94-80bb-d3767a8fb333 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:01.123466 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:01.156613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1070/2152] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:20:01 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:01.158784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ab302-c7c8-44fb-916d-478a5984c8d1 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:01.171122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ab302-c7c8-44fb-916d-478a5984c8d1 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:01.235752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7549-6786-40cb-b8db-3968e122e1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:01.236719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7549-6786-40cb-b8db-3968e122e1da admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:01.236719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7549-6786-40cb-b8db-3968e122e1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:01.237461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7549-6786-40cb-b8db-3968e122e1da admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:01.237862 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f7549-6786-40cb-b8db-3968e122e1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:01.240240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f7549-6786-40cb-b8db-3968e122e1da admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:01.263420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1083/2153] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:20:01 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:01.269236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64031c-e0be-4b94-80bb-d3767a8fb333 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:01.280400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64031c-e0be-4b94-80bb-d3767a8fb333 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:01.286923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04659254-9c0b-4fbb-a3d2-21e3e6032cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:01.287536 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659254-9c0b-4fbb-a3d2-21e3e6032cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:01.288086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659254-9c0b-4fbb-a3d2-21e3e6032cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:01.288409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04659254-9c0b-4fbb-a3d2-21e3e6032cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:01.289019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04659254-9c0b-4fbb-a3d2-21e3e6032cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:01.290629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04659254-9c0b-4fbb-a3d2-21e3e6032cb7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:01.301798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1071/2154] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:01.306181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7549-6786-40cb-b8db-3968e122e1da admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:01.320893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f7549-6786-40cb-b8db-3968e122e1da admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:01.325880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed731714-798b-4f64-a864-5ee2a8b60c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:01.327256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed731714-798b-4f64-a864-5ee2a8b60c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:01.327256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed731714-798b-4f64-a864-5ee2a8b60c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:01.327256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed731714-798b-4f64-a864-5ee2a8b60c7f admin admin] PATH_INFO: `/v3/projects/6c150bb6c92d45cc938686d850a600dc/users/e92196132b2d4d93b3f9de38d10b8380/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:01.344412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed731714-798b-4f64-a864-5ee2a8b60c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c150bb6c92d45cc938686d850a600dc, user_id=e92196132b2d4d93b3f9de38d10b8380, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:01.347748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed731714-798b-4f64-a864-5ee2a8b60c7f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:01.361878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1084/2155] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:01 2022] PUT /identity/v3/projects/6c150bb6c92d45cc938686d850a600dc/users/e92196132b2d4d93b3f9de38d10b8380/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:20:01.365984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04659254-9c0b-4fbb-a3d2-21e3e6032cb7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:01.376812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04659254-9c0b-4fbb-a3d2-21e3e6032cb7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:01.458691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd682c1-e6f4-4c20-b6e5-2fec87a474d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:01.459382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd682c1-e6f4-4c20-b6e5-2fec87a474d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:01.459686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd682c1-e6f4-4c20-b6e5-2fec87a474d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:01.459955 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd682c1-e6f4-4c20-b6e5-2fec87a474d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:01.460621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd682c1-e6f4-4c20-b6e5-2fec87a474d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:01.462229 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd682c1-e6f4-4c20-b6e5-2fec87a474d7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:01.474839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1072/2156] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:01.478908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed731714-798b-4f64-a864-5ee2a8b60c7f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:01.489780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed731714-798b-4f64-a864-5ee2a8b60c7f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:01.494401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:01.495057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:01.495338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:01.495674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] PATH_INFO: `/v3/projects/6c150bb6c92d45cc938686d850a600dc/users/e92196132b2d4d93b3f9de38d10b8380/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:01.497627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c150bb6c92d45cc938686d850a600dc, user_id=e92196132b2d4d93b3f9de38d10b8380, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:01.500269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:01.511146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1085/2157] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:01 2022] PUT /identity/v3/projects/6c150bb6c92d45cc938686d850a600dc/users/e92196132b2d4d93b3f9de38d10b8380/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:01.526353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160610c-bb39-4afc-be54-9aca943b3ac6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:01.526353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160610c-bb39-4afc-be54-9aca943b3ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:01.526353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160610c-bb39-4afc-be54-9aca943b3ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:01.558309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5160610c-bb39-4afc-be54-9aca943b3ac6 None None] MFA Rules not processed for user `e92196132b2d4d93b3f9de38d10b8380`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:01.559532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5160610c-bb39-4afc-be54-9aca943b3ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:01.632198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1073/2158] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:01.646875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:01.649471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:01.718620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495d88-0179-4b49-931c-6604e1c90556 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:01.723461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b99f3-6d73-4183-b457-fb70e4a59b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:01.724205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b99f3-6d73-4183-b457-fb70e4a59b56 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:01.724617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b99f3-6d73-4183-b457-fb70e4a59b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:01.724778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b99f3-6d73-4183-b457-fb70e4a59b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:01.734632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b99f3-6d73-4183-b457-fb70e4a59b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:01.747289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b99f3-6d73-4183-b457-fb70e4a59b56 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:01.747289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1086/2159] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:02.153789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160610c-bb39-4afc-be54-9aca943b3ac6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:02.220401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160610c-bb39-4afc-be54-9aca943b3ac6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:02.225068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:02.225840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:02.226511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:02.226683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:02.227048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] PATH_INFO: `/v3/users/7534951b063c4adcbf14b08d7452b824` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:02.228292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7534951b063c4adcbf14b08d7452b824)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:02.230814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:02.287512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7534951b063c4adcbf14b08d7452b824'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.302288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7534951b063c4adcbf14b08d7452b824'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.303044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7534951b063c4adcbf14b08d7452b824'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.303654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7534951b063c4adcbf14b08d7452b824'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.311374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7534951b063c4adcbf14b08d7452b824'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.328623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1074/2160] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:20:02 2022] DELETE /identity/v3/users/7534951b063c4adcbf14b08d7452b824 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:02.333911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca742c5d-fa1c-4562-a259-02281178fe05 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:02.334203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca742c5d-fa1c-4562-a259-02281178fe05 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:02.334497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca742c5d-fa1c-4562-a259-02281178fe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:02.353994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-ca742c5d-fa1c-4562-a259-02281178fe05 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:02.355367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ca742c5d-fa1c-4562-a259-02281178fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:02.386800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:02.418321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1087/2161] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:02.429333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:02.430782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:02.491017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53e2b2-cb54-4971-843f-fb40471b94e8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:02.494654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e7b1e-0352-487d-a686-c67b4096f63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:02.495262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e7b1e-0352-487d-a686-c67b4096f63f service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:02.495536 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e7b1e-0352-487d-a686-c67b4096f63f service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:02.495810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e7b1e-0352-487d-a686-c67b4096f63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:02.505027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e7b1e-0352-487d-a686-c67b4096f63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:02.506470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e7b1e-0352-487d-a686-c67b4096f63f service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:02.516266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1075/2162] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:02.807114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca742c5d-fa1c-4562-a259-02281178fe05 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:02.808813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ca742c5d-fa1c-4562-a259-02281178fe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:02.821648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca742c5d-fa1c-4562-a259-02281178fe05 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:02.826466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:02.827353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:02.828255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:02.828722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] PATH_INFO: `/v3/projects/0d33a0eab9ea4056ad9a29a989d7ad34` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:02.830446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d33a0eab9ea4056ad9a29a989d7ad34)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:02.832498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:02.880690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d33a0eab9ea4056ad9a29a989d7ad34', 'request_id': 'req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.886803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d33a0eab9ea4056ad9a29a989d7ad34', 'request_id': 'req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.893683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d33a0eab9ea4056ad9a29a989d7ad34', 'request_id': 'req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.906228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] The token cache is being invalidate because project 0d33a0eab9ea4056ad9a29a989d7ad34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:20:02.906574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.907330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:02.909417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1088/2163] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:20:02 2022] DELETE /identity/v3/projects/0d33a0eab9ea4056ad9a29a989d7ad34 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:02.913729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e7b1e-0352-487d-a686-c67b4096f63f service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:02.923380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e7b1e-0352-487d-a686-c67b4096f63f service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:02.986589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:02.987317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:02.987603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:02.987879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] PATH_INFO: `/v3/users/e92196132b2d4d93b3f9de38d10b8380` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:02.989046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e92196132b2d4d93b3f9de38d10b8380)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:02.990964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:03.013562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:03.038996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e92196132b2d4d93b3f9de38d10b8380'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.051279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e92196132b2d4d93b3f9de38d10b8380'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.051947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e92196132b2d4d93b3f9de38d10b8380'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.052548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e92196132b2d4d93b3f9de38d10b8380'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.059800 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1c3cec4d-5aa2-4447-b84f-c1b9be0b4507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e92196132b2d4d93b3f9de38d10b8380'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.073147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1076/2164] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:20:02 2022] DELETE /identity/v3/users/e92196132b2d4d93b3f9de38d10b8380 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:03.343873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:03.345409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:03.359763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e67ba-739c-4f9d-b7e3-7c9c9350c6ed admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:03.419267 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:03.419968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.420251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.420523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] PATH_INFO: `/v3/projects/6c150bb6c92d45cc938686d850a600dc` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.421684 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c150bb6c92d45cc938686d850a600dc)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:03.423646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:03.469669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c150bb6c92d45cc938686d850a600dc', 'request_id': 'req-0984e497-5995-469e-a4fc-eda695915415'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.476887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c150bb6c92d45cc938686d850a600dc', 'request_id': 'req-0984e497-5995-469e-a4fc-eda695915415'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.481435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:03.482595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c150bb6c92d45cc938686d850a600dc', 'request_id': 'req-0984e497-5995-469e-a4fc-eda695915415'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.495095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] The token cache is being invalidate because project 6c150bb6c92d45cc938686d850a600dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:20:03.495387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.496077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0984e497-5995-469e-a4fc-eda695915415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:03.497717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1089/2165] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:20:03 2022] DELETE /identity/v3/projects/6c150bb6c92d45cc938686d850a600dc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:20:03.502563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da273c8-ca0f-4787-998d-0ce0c8a564eb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.503054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da273c8-ca0f-4787-998d-0ce0c8a564eb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.503160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da273c8-ca0f-4787-998d-0ce0c8a564eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.523845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2da273c8-ca0f-4787-998d-0ce0c8a564eb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:03.525112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2da273c8-ca0f-4787-998d-0ce0c8a564eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:03.586068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1077/2166] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:03.594828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1e588-a399-46c6-a121-258fc3348f7a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.595056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1e588-a399-46c6-a121-258fc3348f7a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.595382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1e588-a399-46c6-a121-258fc3348f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.615207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-84e1e588-a399-46c6-a121-258fc3348f7a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:03.616423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-84e1e588-a399-46c6-a121-258fc3348f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:03.624268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1090/2167] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:03.629427 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da273c8-ca0f-4787-998d-0ce0c8a564eb None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:03.639188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da273c8-ca0f-4787-998d-0ce0c8a564eb None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:03.644376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:03.645000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.645277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.645545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.646184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:20:03.646483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:03.648082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:03.659495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1078/2168] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:20:03.664207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1e588-a399-46c6-a121-258fc3348f7a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:03.673625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1e588-a399-46c6-a121-258fc3348f7a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:03.678732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9a8c3-f7c0-4954-95de-16adb5002b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:03.679318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9a8c3-f7c0-4954-95de-16adb5002b90 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.679601 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9a8c3-f7c0-4954-95de-16adb5002b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.679874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9a8c3-f7c0-4954-95de-16adb5002b90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.680518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9a8c3-f7c0-4954-95de-16adb5002b90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:03.682474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9a8c3-f7c0-4954-95de-16adb5002b90 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:03.713046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1091/2169] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:20:03 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:03.717785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:03.729347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5a5df-55e9-42b8-a78b-42815a13ad7b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:03.786767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070d5bf-f0ab-4821-991b-ae27864c55aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:03.787378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d5bf-f0ab-4821-991b-ae27864c55aa admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.787916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d5bf-f0ab-4821-991b-ae27864c55aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.788184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d5bf-f0ab-4821-991b-ae27864c55aa admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.788869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070d5bf-f0ab-4821-991b-ae27864c55aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:03.790823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070d5bf-f0ab-4821-991b-ae27864c55aa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:03.816977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1079/2170] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:20:03 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:03.818161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9a8c3-f7c0-4954-95de-16adb5002b90 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:03.828866 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9a8c3-f7c0-4954-95de-16adb5002b90 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:03.834812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56050c-faa1-4a9a-8317-6103e7d87e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:03.835419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56050c-faa1-4a9a-8317-6103e7d87e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.835692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56050c-faa1-4a9a-8317-6103e7d87e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.835982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56050c-faa1-4a9a-8317-6103e7d87e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.836648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56050c-faa1-4a9a-8317-6103e7d87e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:03.838293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56050c-faa1-4a9a-8317-6103e7d87e74 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:03.848255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1092/2171] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:20:03.852109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070d5bf-f0ab-4821-991b-ae27864c55aa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:03.861895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070d5bf-f0ab-4821-991b-ae27864c55aa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:03.866448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957f874-0743-475c-bbed-4593c3b80ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:03.867028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f874-0743-475c-bbed-4593c3b80ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.867305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f874-0743-475c-bbed-4593c3b80ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.867574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f874-0743-475c-bbed-4593c3b80ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.868204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957f874-0743-475c-bbed-4593c3b80ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:03.869843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957f874-0743-475c-bbed-4593c3b80ae8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:03.881451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1080/2172] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:03.885519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56050c-faa1-4a9a-8317-6103e7d87e74 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:03.896046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56050c-faa1-4a9a-8317-6103e7d87e74 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:03.900642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:03.901286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.901568 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.901841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] PATH_INFO: `/v3/projects/f4d3b66fea164c2b93aa58fb75613abc/users/3be867e3ddc84fc8a8aac9aa81c433b5/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.917538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d3b66fea164c2b93aa58fb75613abc, user_id=3be867e3ddc84fc8a8aac9aa81c433b5, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:03.920178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:03.933735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1093/2173] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:03 2022] PUT /identity/v3/projects/f4d3b66fea164c2b93aa58fb75613abc/users/3be867e3ddc84fc8a8aac9aa81c433b5/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:03.938779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de269279-b891-4d43-9ff1-7a45521f06ef None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:03.939062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de269279-b891-4d43-9ff1-7a45521f06ef None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:03.939939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de269279-b891-4d43-9ff1-7a45521f06ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:03.947808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:03.961323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-de269279-b891-4d43-9ff1-7a45521f06ef None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:03.962551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-de269279-b891-4d43-9ff1-7a45521f06ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:04.025079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1081/2174] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:04.034835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:04.036541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:04.093249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140b052-c70c-433b-b99c-4d0db2da27ba admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:04.096962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc73de-8705-4f47-a8b2-6f80d8eb1e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:04.097555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc73de-8705-4f47-a8b2-6f80d8eb1e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:04.097829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc73de-8705-4f47-a8b2-6f80d8eb1e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:04.098098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc73de-8705-4f47-a8b2-6f80d8eb1e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:04.107069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc73de-8705-4f47-a8b2-6f80d8eb1e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:04.108435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc73de-8705-4f47-a8b2-6f80d8eb1e49 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:04.118444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1094/2175] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:04.740976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703048d-1d85-4997-bab8-086c552d32b3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:04.741522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703048d-1d85-4997-bab8-086c552d32b3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:04.741522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703048d-1d85-4997-bab8-086c552d32b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:04.770141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6703048d-1d85-4997-bab8-086c552d32b3 None None] MFA Rules not processed for user `3be867e3ddc84fc8a8aac9aa81c433b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:04.771355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6703048d-1d85-4997-bab8-086c552d32b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:04.855832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1082/2176] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:04.860555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc73de-8705-4f47-a8b2-6f80d8eb1e49 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:04.870693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc73de-8705-4f47-a8b2-6f80d8eb1e49 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:04.875632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29171102-645a-4b67-8d80-c46ed2500acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:04.876250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171102-645a-4b67-8d80-c46ed2500acd admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:04.876546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171102-645a-4b67-8d80-c46ed2500acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:04.876882 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171102-645a-4b67-8d80-c46ed2500acd admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:04.877643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29171102-645a-4b67-8d80-c46ed2500acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:04.879590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29171102-645a-4b67-8d80-c46ed2500acd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:04.913354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1095/2177] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:20:04 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:04.914498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703048d-1d85-4997-bab8-086c552d32b3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:04.925865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703048d-1d85-4997-bab8-086c552d32b3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:04.995672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67127b9-9a4c-4453-8158-b87aaff5cedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:04.996777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67127b9-9a4c-4453-8158-b87aaff5cedd admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:04.997470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67127b9-9a4c-4453-8158-b87aaff5cedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:04.998394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67127b9-9a4c-4453-8158-b87aaff5cedd admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:05.000467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67127b9-9a4c-4453-8158-b87aaff5cedd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:05.004255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67127b9-9a4c-4453-8158-b87aaff5cedd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:05.035652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1083/2178] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:20:04 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:05.040137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29171102-645a-4b67-8d80-c46ed2500acd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:05.050913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29171102-645a-4b67-8d80-c46ed2500acd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:05.055569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac54cec-0c95-49ba-afaa-ee7d163b34bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:05.056234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac54cec-0c95-49ba-afaa-ee7d163b34bb admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:05.056622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac54cec-0c95-49ba-afaa-ee7d163b34bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:05.056961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac54cec-0c95-49ba-afaa-ee7d163b34bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:05.057694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac54cec-0c95-49ba-afaa-ee7d163b34bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:05.059363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac54cec-0c95-49ba-afaa-ee7d163b34bb admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:05.072040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1096/2179] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:05.076322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67127b9-9a4c-4453-8158-b87aaff5cedd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:05.087824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67127b9-9a4c-4453-8158-b87aaff5cedd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:05.092527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7446ab-9b7f-4607-8336-db9420b8c902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:05.093202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7446ab-9b7f-4607-8336-db9420b8c902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:05.093478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7446ab-9b7f-4607-8336-db9420b8c902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:05.093743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7446ab-9b7f-4607-8336-db9420b8c902 admin admin] PATH_INFO: `/v3/projects/3d7eb3434d064a7dbaca34624121c5a4/users/332048fb79834d4a9f1a7e246a34233b/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:05.108404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7446ab-9b7f-4607-8336-db9420b8c902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7eb3434d064a7dbaca34624121c5a4, user_id=332048fb79834d4a9f1a7e246a34233b, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:05.111110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7446ab-9b7f-4607-8336-db9420b8c902 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:05.119908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:05.125174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1084/2180] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:05 2022] PUT /identity/v3/projects/3d7eb3434d064a7dbaca34624121c5a4/users/332048fb79834d4a9f1a7e246a34233b/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:05.131084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac54cec-0c95-49ba-afaa-ee7d163b34bb admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:05.141166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac54cec-0c95-49ba-afaa-ee7d163b34bb admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:05.196450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:05.204316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae52d4f-f7d0-4675-b1c5-e12a10b6230c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:05.205800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52d4f-f7d0-4675-b1c5-e12a10b6230c admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:05.206694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52d4f-f7d0-4675-b1c5-e12a10b6230c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:05.207421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52d4f-f7d0-4675-b1c5-e12a10b6230c admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:05.208114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae52d4f-f7d0-4675-b1c5-e12a10b6230c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:05.209873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae52d4f-f7d0-4675-b1c5-e12a10b6230c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:05.223226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1097/2181] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:05.224382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7446ab-9b7f-4607-8336-db9420b8c902 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:05.235041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7446ab-9b7f-4607-8336-db9420b8c902 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:05.239762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:05.240414 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:05.240733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:05.241030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] PATH_INFO: `/v3/projects/3d7eb3434d064a7dbaca34624121c5a4/users/332048fb79834d4a9f1a7e246a34233b/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:05.242906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7eb3434d064a7dbaca34624121c5a4, user_id=332048fb79834d4a9f1a7e246a34233b, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:05.245591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:05.256255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1085/2182] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:05 2022] PUT /identity/v3/projects/3d7eb3434d064a7dbaca34624121c5a4/users/332048fb79834d4a9f1a7e246a34233b/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:05.919852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec95f2d-8712-455a-9884-190c147d4350 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:05.920356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec95f2d-8712-455a-9884-190c147d4350 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:05.920451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec95f2d-8712-455a-9884-190c147d4350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:05.949381 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-3ec95f2d-8712-455a-9884-190c147d4350 None None] MFA Rules not processed for user `332048fb79834d4a9f1a7e246a34233b`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:05.950618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3ec95f2d-8712-455a-9884-190c147d4350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:06.025255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1098/2183] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:06.037988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:06.039714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:06.101068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9fd78-deff-4a21-83ef-5eb790359b94 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:06.104725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b6639-9e20-454e-8e0a-0c61d790185f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:06.105328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b6639-9e20-454e-8e0a-0c61d790185f service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:06.105607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b6639-9e20-454e-8e0a-0c61d790185f service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:06.105877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b6639-9e20-454e-8e0a-0c61d790185f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:06.164758 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b6639-9e20-454e-8e0a-0c61d790185f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:06.166205 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b6639-9e20-454e-8e0a-0c61d790185f service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:06.176109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1086/2184] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:07.536491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec95f2d-8712-455a-9884-190c147d4350 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:07.538551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3ec95f2d-8712-455a-9884-190c147d4350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:07.551640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec95f2d-8712-455a-9884-190c147d4350 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:07.555616 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:07.556511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:07.556714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:07.556990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:07.567232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:07.568780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:07.584911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1099/2185] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:07.668594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0561ea-329d-47b9-9f90-0efd87ea3b9f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:07.669523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0561ea-329d-47b9-9f90-0efd87ea3b9f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:07.670073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0561ea-329d-47b9-9f90-0efd87ea3b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:07.692096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4a0561ea-329d-47b9-9f90-0efd87ea3b9f None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:07.693571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4a0561ea-329d-47b9-9f90-0efd87ea3b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:07.760476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1087/2186] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:20:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:07.819495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:07.820495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:07.902346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c90a7-009d-40ef-bcfd-a124552e3224 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:07.907409 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75ff0d-d494-439c-b6b8-83f967a4e7b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:07.908673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75ff0d-d494-439c-b6b8-83f967a4e7b6 service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:07.908950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75ff0d-d494-439c-b6b8-83f967a4e7b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:07.909228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75ff0d-d494-439c-b6b8-83f967a4e7b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:07.919488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75ff0d-d494-439c-b6b8-83f967a4e7b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:07.921861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75ff0d-d494-439c-b6b8-83f967a4e7b6 service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:07.930148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1100/2187] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:20:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:20:10.694563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0561ea-329d-47b9-9f90-0efd87ea3b9f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:10.696729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4a0561ea-329d-47b9-9f90-0efd87ea3b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:10.777628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:10.834949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0561ea-329d-47b9-9f90-0efd87ea3b9f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:10.841359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:10.842135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:10.842419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:10.842689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] PATH_INFO: `/v3/users/83035f93b62146d4bcb34e8ef4be5ab4` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:10.843873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83035f93b62146d4bcb34e8ef4be5ab4)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:10.849176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:10.959215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83035f93b62146d4bcb34e8ef4be5ab4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:10.982711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83035f93b62146d4bcb34e8ef4be5ab4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:10.984607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83035f93b62146d4bcb34e8ef4be5ab4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:10.985492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83035f93b62146d4bcb34e8ef4be5ab4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:10.998688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83035f93b62146d4bcb34e8ef4be5ab4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:11.019549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1088/2188] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:20:10 2022] DELETE /identity/v3/users/83035f93b62146d4bcb34e8ef4be5ab4 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:11.023930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb35c20-212b-491d-8cf7-a3638e0beca9 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:11.024988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb35c20-212b-491d-8cf7-a3638e0beca9 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:11.026077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb35c20-212b-491d-8cf7-a3638e0beca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:11.061486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-cbb35c20-212b-491d-8cf7-a3638e0beca9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:11.063247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cbb35c20-212b-491d-8cf7-a3638e0beca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:11.163208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:11.186222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1101/2189] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:20:11.198090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:11.199300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:11.303123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199cd06-5423-40f0-bf80-35591c03ec25 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:11.314115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22af9b-7427-4a0b-b05f-d4ced7a456be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:11.314115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22af9b-7427-4a0b-b05f-d4ced7a456be service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:11.314507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22af9b-7427-4a0b-b05f-d4ced7a456be service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:11.316749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22af9b-7427-4a0b-b05f-d4ced7a456be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:11.328088 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22af9b-7427-4a0b-b05f-d4ced7a456be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:11.329510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22af9b-7427-4a0b-b05f-d4ced7a456be service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:11.343958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1089/2190] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:11.711050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb35c20-212b-491d-8cf7-a3638e0beca9 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:11.712823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cbb35c20-212b-491d-8cf7-a3638e0beca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:11.733328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb35c20-212b-491d-8cf7-a3638e0beca9 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:11.740257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:11.741191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:11.741441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:11.741712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] PATH_INFO: `/v3/projects/0ebfb6d4850046f78352f81dcc8f6660` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:11.742833 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebfb6d4850046f78352f81dcc8f6660)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:11.750369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:11.807481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebfb6d4850046f78352f81dcc8f6660', 'request_id': 'req-719982a1-59cb-4783-810b-8ed548ae4192'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:11.816404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebfb6d4850046f78352f81dcc8f6660', 'request_id': 'req-719982a1-59cb-4783-810b-8ed548ae4192'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:11.823111 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebfb6d4850046f78352f81dcc8f6660', 'request_id': 'req-719982a1-59cb-4783-810b-8ed548ae4192'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:11.839463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] The token cache is being invalidate because project 0ebfb6d4850046f78352f81dcc8f6660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:20:11.839782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:11.840521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-719982a1-59cb-4783-810b-8ed548ae4192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:11.842387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1102/2191] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:20:11 2022] DELETE /identity/v3/projects/0ebfb6d4850046f78352f81dcc8f6660 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:11.847136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45674654-f634-41d6-ab31-0175ac2d010a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:11.847705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45674654-f634-41d6-ab31-0175ac2d010a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:11.847799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45674654-f634-41d6-ab31-0175ac2d010a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:11.869396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-45674654-f634-41d6-ab31-0175ac2d010a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:11.870627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-45674654-f634-41d6-ab31-0175ac2d010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:11.936273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1090/2192] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:11.945479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a305-b73c-46e3-929d-4faeeef20dae None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:11.945853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a305-b73c-46e3-929d-4faeeef20dae None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:11.946197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a305-b73c-46e3-929d-4faeeef20dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:11.966989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-2058a305-b73c-46e3-929d-4faeeef20dae None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:11.968232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2058a305-b73c-46e3-929d-4faeeef20dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:11.976341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1103/2193] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:11.980698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45674654-f634-41d6-ab31-0175ac2d010a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:11.991093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45674654-f634-41d6-ab31-0175ac2d010a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:11.996034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:11.996667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:11.996959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:11.997232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:11.997881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:20:11.998187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:11.999787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:12.002816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:12.014572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1091/2194] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:20:12.019459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058a305-b73c-46e3-929d-4faeeef20dae None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:12.029134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058a305-b73c-46e3-929d-4faeeef20dae None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:12.034013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d36f8-f486-4475-9a7e-f383daffb92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:12.034590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d36f8-f486-4475-9a7e-f383daffb92d admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:12.034867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d36f8-f486-4475-9a7e-f383daffb92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:12.035328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d36f8-f486-4475-9a7e-f383daffb92d admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:12.035970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d36f8-f486-4475-9a7e-f383daffb92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:12.038188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d36f8-f486-4475-9a7e-f383daffb92d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:12.066831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1104/2195] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:20:12 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:12.071186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:12.085271 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9946dd-88be-4209-80d9-0b1a9be6bf2c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:12.148929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505fe5f-8e9e-421f-a0b6-c9c030745c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:12.149648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505fe5f-8e9e-421f-a0b6-c9c030745c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:12.150021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505fe5f-8e9e-421f-a0b6-c9c030745c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:12.150316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505fe5f-8e9e-421f-a0b6-c9c030745c6f admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:12.151014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505fe5f-8e9e-421f-a0b6-c9c030745c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:12.153061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505fe5f-8e9e-421f-a0b6-c9c030745c6f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:12.178722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1092/2196] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:20:12 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:20:12.182709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d36f8-f486-4475-9a7e-f383daffb92d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:12.194512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d36f8-f486-4475-9a7e-f383daffb92d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:12.199651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86e409-dd90-4ff7-9a93-67d975a25cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:12.200289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86e409-dd90-4ff7-9a93-67d975a25cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:12.200597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86e409-dd90-4ff7-9a93-67d975a25cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:12.200866 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86e409-dd90-4ff7-9a93-67d975a25cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:12.201525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86e409-dd90-4ff7-9a93-67d975a25cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:12.203124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86e409-dd90-4ff7-9a93-67d975a25cd6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:12.214049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1105/2197] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:12.221584 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505fe5f-8e9e-421f-a0b6-c9c030745c6f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:12.234258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505fe5f-8e9e-421f-a0b6-c9c030745c6f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:12.240143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db864fa-ccf9-41c0-9218-719cd2fd47bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:12.241087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db864fa-ccf9-41c0-9218-719cd2fd47bf admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:12.241435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db864fa-ccf9-41c0-9218-719cd2fd47bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:12.241656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db864fa-ccf9-41c0-9218-719cd2fd47bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:12.242729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db864fa-ccf9-41c0-9218-719cd2fd47bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:12.244101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db864fa-ccf9-41c0-9218-719cd2fd47bf admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:12.259694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1093/2198] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:20:12.260929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86e409-dd90-4ff7-9a93-67d975a25cd6 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:12.272792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86e409-dd90-4ff7-9a93-67d975a25cd6 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:12.277436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:12.278120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:12.278398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:12.278671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] PATH_INFO: `/v3/projects/ffaeee7ebbe34372a16f7a8b7b309e03/users/668428c74f98417987ee6c54ee9181d2/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:12.295759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffaeee7ebbe34372a16f7a8b7b309e03, user_id=668428c74f98417987ee6c54ee9181d2, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:12.298502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:12.312060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1106/2199] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:12 2022] PUT /identity/v3/projects/ffaeee7ebbe34372a16f7a8b7b309e03/users/668428c74f98417987ee6c54ee9181d2/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:12.322600 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322312c-f584-4e9a-aff0-d72cf22257b2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:12.322919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322312c-f584-4e9a-aff0-d72cf22257b2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:12.323195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322312c-f584-4e9a-aff0-d72cf22257b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:12.352394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9322312c-f584-4e9a-aff0-d72cf22257b2 None None] MFA Rules not processed for user `668428c74f98417987ee6c54ee9181d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:12.353662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9322312c-f584-4e9a-aff0-d72cf22257b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:12.426859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1094/2200] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:12.436360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:12.438361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:12.509003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab1dc9-b223-435f-b3e9-c7327817bf6f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:12.512664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3a4a5-11e5-4310-9fd9-7520ad7f017c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6c90a2b95c498494a73b43e7138905', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:12.513268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3a4a5-11e5-4310-9fd9-7520ad7f017c service glance] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:12.513544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3a4a5-11e5-4310-9fd9-7520ad7f017c service glance] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:12.513813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3a4a5-11e5-4310-9fd9-7520ad7f017c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:12.524164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3a4a5-11e5-4310-9fd9-7520ad7f017c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:12.525698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3a4a5-11e5-4310-9fd9-7520ad7f017c service glance] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:12.536427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1107/2201] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:22.242245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd80bca-5b8c-4658-903f-4e7d3915a28f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:22.243210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd80bca-5b8c-4658-903f-4e7d3915a28f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:22.243210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd80bca-5b8c-4658-903f-4e7d3915a28f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:22.244349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1095/2202] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:20:22.265588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923cfa2-69e3-420c-88af-b22c8a8ab386 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:22.266641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923cfa2-69e3-420c-88af-b22c8a8ab386 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:22.266641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923cfa2-69e3-420c-88af-b22c8a8ab386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:22.291627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f923cfa2-69e3-420c-88af-b22c8a8ab386 None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:22.293717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f923cfa2-69e3-420c-88af-b22c8a8ab386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:22.306930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:22.418711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:22.488058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1108/2203] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:20:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:22.779502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123818a8-3886-4e31-9bed-82017dc3f38f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:22.779502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123818a8-3886-4e31-9bed-82017dc3f38f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:22.779502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123818a8-3886-4e31-9bed-82017dc3f38f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:22.781079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1096/2204] 10.222.0.95 () {54 vars in 961 bytes} [Wed Aug 31 19:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:20:22.802709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44164137-5479-4786-a726-1c2b6f4799dd None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:22.803884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44164137-5479-4786-a726-1c2b6f4799dd None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:22.804366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44164137-5479-4786-a726-1c2b6f4799dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:22.850515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-44164137-5479-4786-a726-1c2b6f4799dd None None] MFA Rules not processed for user `a39f8675dcfa4988af3a402b32984b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:22.853206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-44164137-5479-4786-a726-1c2b6f4799dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:22.868693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1109/2205] 10.222.0.95 () {58 vars in 1058 bytes} [Wed Aug 31 19:20:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:20:22.887400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123818a8-3886-4e31-9bed-82017dc3f38f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:22.943267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123818a8-3886-4e31-9bed-82017dc3f38f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:22.968122 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92bd99-2f4a-4c45-a396-50a25560297d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:22.970780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92bd99-2f4a-4c45-a396-50a25560297d service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:22.971559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92bd99-2f4a-4c45-a396-50a25560297d service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:22.971973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92bd99-2f4a-4c45-a396-50a25560297d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:22.991146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92bd99-2f4a-4c45-a396-50a25560297d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:22.993840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92bd99-2f4a-4c45-a396-50a25560297d service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:23.006620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1097/2206] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:23.702936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44164137-5479-4786-a726-1c2b6f4799dd None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:23.707829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-44164137-5479-4786-a726-1c2b6f4799dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:23.789579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44164137-5479-4786-a726-1c2b6f4799dd None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:23.794369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:23.795071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:23.795419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:23.795820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] PATH_INFO: `/v3/users/3be867e3ddc84fc8a8aac9aa81c433b5` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:23.797354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be867e3ddc84fc8a8aac9aa81c433b5)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:23.800296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:23.860136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be867e3ddc84fc8a8aac9aa81c433b5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:23.868019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be867e3ddc84fc8a8aac9aa81c433b5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:23.868753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be867e3ddc84fc8a8aac9aa81c433b5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:23.883844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be867e3ddc84fc8a8aac9aa81c433b5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:23.884617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be867e3ddc84fc8a8aac9aa81c433b5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:23.899629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1110/2207] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:20:23 2022] DELETE /identity/v3/users/3be867e3ddc84fc8a8aac9aa81c433b5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:24.297839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92bd99-2f4a-4c45-a396-50a25560297d service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:24.317412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92bd99-2f4a-4c45-a396-50a25560297d service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:24.456868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:24.457570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:24.457860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:24.458495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] PATH_INFO: `/v3/projects/f4d3b66fea164c2b93aa58fb75613abc` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:24.459848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d3b66fea164c2b93aa58fb75613abc)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:24.461921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:24.537218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d3b66fea164c2b93aa58fb75613abc', 'request_id': 'req-9966d89d-0a20-475a-b333-8cd82f59cc8d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.555124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d3b66fea164c2b93aa58fb75613abc', 'request_id': 'req-9966d89d-0a20-475a-b333-8cd82f59cc8d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.569816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d3b66fea164c2b93aa58fb75613abc', 'request_id': 'req-9966d89d-0a20-475a-b333-8cd82f59cc8d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.592374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] The token cache is being invalidate because project f4d3b66fea164c2b93aa58fb75613abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:20:24.592374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.593064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9966d89d-0a20-475a-b333-8cd82f59cc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.594819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1098/2208] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:20:24 2022] DELETE /identity/v3/projects/f4d3b66fea164c2b93aa58fb75613abc => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:24.600112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:24.602373 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:24.619013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02688e-0dcf-403f-9582-4cce2f84e412 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:24.657027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:24.709397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:24.710096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:24.710344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:24.710646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] PATH_INFO: `/v3/users/332048fb79834d4a9f1a7e246a34233b` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:24.711896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=332048fb79834d4a9f1a7e246a34233b)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:24.714198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:24.801114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '332048fb79834d4a9f1a7e246a34233b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.813187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '332048fb79834d4a9f1a7e246a34233b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.814498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '332048fb79834d4a9f1a7e246a34233b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.848747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '332048fb79834d4a9f1a7e246a34233b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.850054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '332048fb79834d4a9f1a7e246a34233b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:24.868180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1111/2209] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:20:24 2022] DELETE /identity/v3/users/332048fb79834d4a9f1a7e246a34233b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:24.984561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c454399-990c-4bbb-b0fb-555c8b12c42a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:24.984865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c454399-990c-4bbb-b0fb-555c8b12c42a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:24.985158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c454399-990c-4bbb-b0fb-555c8b12c42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.030065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6c454399-990c-4bbb-b0fb-555c8b12c42a None None] MFA Rules not processed for user `6e09978ed8b3490db5f92823768f1d15`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:25.031332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6c454399-990c-4bbb-b0fb-555c8b12c42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:25.117734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:25.119374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:25.149137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1099/2210] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:25.216518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08091ea7-73e0-48e5-a60e-a67cc9a825b1 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:25.221741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:25.222448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:25.222761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:25.223036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] PATH_INFO: `/v3/projects/3d7eb3434d064a7dbaca34624121c5a4` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.224221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7eb3434d064a7dbaca34624121c5a4)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:25.226290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:25.277493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:25.283173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7eb3434d064a7dbaca34624121c5a4', 'request_id': 'req-728c3948-a76c-42c6-93b0-5768fabd956a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:25.294482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7eb3434d064a7dbaca34624121c5a4', 'request_id': 'req-728c3948-a76c-42c6-93b0-5768fabd956a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:25.304113 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7eb3434d064a7dbaca34624121c5a4', 'request_id': 'req-728c3948-a76c-42c6-93b0-5768fabd956a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:25.318257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] The token cache is being invalidate because project 3d7eb3434d064a7dbaca34624121c5a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:20:25.318550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:25.319818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-728c3948-a76c-42c6-93b0-5768fabd956a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:25.321438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1112/2211] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:20:25 2022] DELETE /identity/v3/projects/3d7eb3434d064a7dbaca34624121c5a4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:25.326811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71dc4-e39f-47f8-90f8-e2814249ee14 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:25.327337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71dc4-e39f-47f8-90f8-e2814249ee14 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:25.327466 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71dc4-e39f-47f8-90f8-e2814249ee14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.352277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f5e71dc4-e39f-47f8-90f8-e2814249ee14 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:25.354150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f5e71dc4-e39f-47f8-90f8-e2814249ee14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:25.379378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:25.433838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1100/2212] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:25.436020 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e50ab5-09e2-4ec0-a3cd-bbcef372c00e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:25.436313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e50ab5-09e2-4ec0-a3cd-bbcef372c00e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:25.436617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e50ab5-09e2-4ec0-a3cd-bbcef372c00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.463617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-31e50ab5-09e2-4ec0-a3cd-bbcef372c00e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:25.465400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-31e50ab5-09e2-4ec0-a3cd-bbcef372c00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:25.486935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1113/2213] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:20:25.493749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e71dc4-e39f-47f8-90f8-e2814249ee14 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:25.514592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e71dc4-e39f-47f8-90f8-e2814249ee14 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:25.523413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:25.524095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:25.524798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:25.525074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.525742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:20:25.526058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:25.528102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:25.567651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1101/2214] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:20:25.569583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e50ab5-09e2-4ec0-a3cd-bbcef372c00e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:25.586819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e50ab5-09e2-4ec0-a3cd-bbcef372c00e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:25.591866 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adffbe-df06-42ba-a309-c2cf68a3499f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:25.592754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adffbe-df06-42ba-a309-c2cf68a3499f admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:25.593290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adffbe-df06-42ba-a309-c2cf68a3499f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:25.593588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adffbe-df06-42ba-a309-c2cf68a3499f admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.594466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adffbe-df06-42ba-a309-c2cf68a3499f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:25.597095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adffbe-df06-42ba-a309-c2cf68a3499f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:25.635259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1114/2215] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:20:25 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:25.640176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:25.653506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36c7ee-861f-4f1d-82b3-ac57ca7a3460 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:25.774651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff71cf4-bf15-45b0-b9d4-6e186adf298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:25.775261 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff71cf4-bf15-45b0-b9d4-6e186adf298c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:25.775544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff71cf4-bf15-45b0-b9d4-6e186adf298c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:25.776024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff71cf4-bf15-45b0-b9d4-6e186adf298c admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.777178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff71cf4-bf15-45b0-b9d4-6e186adf298c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:25.779541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff71cf4-bf15-45b0-b9d4-6e186adf298c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:25.810642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1102/2216] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:20:25 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:25.815026 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adffbe-df06-42ba-a309-c2cf68a3499f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:25.826585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adffbe-df06-42ba-a309-c2cf68a3499f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:25.844397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca1d59-71bf-496a-a79e-7714e5d6c9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:25.845672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca1d59-71bf-496a-a79e-7714e5d6c9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:25.846024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca1d59-71bf-496a-a79e-7714e5d6c9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:25.846300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca1d59-71bf-496a-a79e-7714e5d6c9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.847102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca1d59-71bf-496a-a79e-7714e5d6c9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:25.848775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca1d59-71bf-496a-a79e-7714e5d6c9b9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:25.888584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1115/2217] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:25.890371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff71cf4-bf15-45b0-b9d4-6e186adf298c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:25.916806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff71cf4-bf15-45b0-b9d4-6e186adf298c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:25.927809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81853f-70bd-401f-87ec-fcb8f743588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:25.928467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81853f-70bd-401f-87ec-fcb8f743588d admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:25.929180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81853f-70bd-401f-87ec-fcb8f743588d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:25.929470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81853f-70bd-401f-87ec-fcb8f743588d admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:25.930134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81853f-70bd-401f-87ec-fcb8f743588d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:25.934464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81853f-70bd-401f-87ec-fcb8f743588d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:25.953738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1103/2218] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:25.958921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca1d59-71bf-496a-a79e-7714e5d6c9b9 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:25.978463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca1d59-71bf-496a-a79e-7714e5d6c9b9 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:26.002580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:26.004807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:26.005910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:26.006107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] PATH_INFO: `/v3/projects/fc4ee3fd0f8f42e491e3c47cb6c27899/users/00785d2c9c6d440b8c34c7f9cc664e1f/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:26.053433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4ee3fd0f8f42e491e3c47cb6c27899, user_id=00785d2c9c6d440b8c34c7f9cc664e1f, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:26.065856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:26.120649 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1116/2219] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:25 2022] PUT /identity/v3/projects/fc4ee3fd0f8f42e491e3c47cb6c27899/users/00785d2c9c6d440b8c34c7f9cc664e1f/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:26.128607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ece1c-05c4-4fa0-a613-dd25cb3b6977 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:26.128979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ece1c-05c4-4fa0-a613-dd25cb3b6977 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:26.129419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ece1c-05c4-4fa0-a613-dd25cb3b6977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:26.194030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-030ece1c-05c4-4fa0-a613-dd25cb3b6977 None None] MFA Rules not processed for user `00785d2c9c6d440b8c34c7f9cc664e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:26.195703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-030ece1c-05c4-4fa0-a613-dd25cb3b6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:26.319625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1104/2220] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:26.328561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:26.334473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:26.418024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0c19-bb6a-40a7-9d77-3c5f3459b76c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:26.421976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f8fb-3697-41a5-9c17-ebebd471dcec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:26.422582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f8fb-3697-41a5-9c17-ebebd471dcec service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:26.423393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f8fb-3697-41a5-9c17-ebebd471dcec service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:26.423692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f8fb-3697-41a5-9c17-ebebd471dcec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:26.432908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f8fb-3697-41a5-9c17-ebebd471dcec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:26.434647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6f8fb-3697-41a5-9c17-ebebd471dcec service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:26.447187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1117/2221] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:36.310746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:36.441837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:48.216837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ece1c-05c4-4fa0-a613-dd25cb3b6977 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:48.221015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-030ece1c-05c4-4fa0-a613-dd25cb3b6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:48.363816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ece1c-05c4-4fa0-a613-dd25cb3b6977 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:48.378506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:48.380958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:48.383700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:48.383885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:48.387321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:48.401580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:48.403211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:48.420528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1105/2222] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:50.619674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28150e6-b3f9-444f-a6ca-22a034a688c6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:50.620306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28150e6-b3f9-444f-a6ca-22a034a688c6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:50.620444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28150e6-b3f9-444f-a6ca-22a034a688c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:50.649034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b28150e6-b3f9-444f-a6ca-22a034a688c6 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:50.650759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b28150e6-b3f9-444f-a6ca-22a034a688c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:50.803655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1118/2223] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:20:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:50.827108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:50.829171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:50.969231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f1fee-aefd-4eb3-8dcb-b42ae23a259d service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:50.969231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6221-9e5e-48c5-bb6b-06a3ba91d326 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:50.969231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6221-9e5e-48c5-bb6b-06a3ba91d326 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:50.969231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6221-9e5e-48c5-bb6b-06a3ba91d326 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:50.970212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6221-9e5e-48c5-bb6b-06a3ba91d326 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:50.975973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f6221-9e5e-48c5-bb6b-06a3ba91d326 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:50.978543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f6221-9e5e-48c5-bb6b-06a3ba91d326 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:50.991080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1106/2224] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:20:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:20:56.031254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28150e6-b3f9-444f-a6ca-22a034a688c6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:56.238997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28150e6-b3f9-444f-a6ca-22a034a688c6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:56.247241 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:56.249266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:56.250260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:56.250959 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] PATH_INFO: `/v3/users/00785d2c9c6d440b8c34c7f9cc664e1f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:56.253204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00785d2c9c6d440b8c34c7f9cc664e1f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:56.256413 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:56.297045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc7f3-c730-4e38-9099-38570f0cf5d9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:56.298286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc7f3-c730-4e38-9099-38570f0cf5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:56.298656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dc7f3-c730-4e38-9099-38570f0cf5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:56.333006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c40dc7f3-c730-4e38-9099-38570f0cf5d9 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:56.336915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c40dc7f3-c730-4e38-9099-38570f0cf5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:56.356842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1107/2225] 10.222.0.95 () {58 vars in 967 bytes} [Wed Aug 31 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:56.423557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00785d2c9c6d440b8c34c7f9cc664e1f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:56.423879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dc7f3-c730-4e38-9099-38570f0cf5d9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:56.444966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00785d2c9c6d440b8c34c7f9cc664e1f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:56.446893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00785d2c9c6d440b8c34c7f9cc664e1f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:56.448347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dc7f3-c730-4e38-9099-38570f0cf5d9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:56.456102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:56.457171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:56.457742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:56.458504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:56.471449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:56.479567 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00785d2c9c6d440b8c34c7f9cc664e1f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:56.481757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0e72d2c3-bb90-4697-bebb-038d28eed10f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00785d2c9c6d440b8c34c7f9cc664e1f'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:56.497009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:56.499839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:56.503306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1119/2226] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:20:56 2022] DELETE /identity/v3/users/00785d2c9c6d440b8c34c7f9cc664e1f => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:20:56.506280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809945-d63f-48f0-a1a9-42ba19d2f88a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:56.506280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809945-d63f-48f0-a1a9-42ba19d2f88a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:56.506280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809945-d63f-48f0-a1a9-42ba19d2f88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:56.527485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1108/2227] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:20:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:20:56.548590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a9809945-d63f-48f0-a1a9-42ba19d2f88a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:56.551754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a9809945-d63f-48f0-a1a9-42ba19d2f88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:56.588187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:56.736622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1120/2228] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:56.743690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:56.745237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:56.889687 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa750d21-1535-4d3e-a793-87699a117fdd service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:56.940471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:57.030131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e0884-37a9-4024-a2a2-55eb10dc76ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:57.030860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0884-37a9-4024-a2a2-55eb10dc76ab service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:57.031280 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0884-37a9-4024-a2a2-55eb10dc76ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:57.031643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e0884-37a9-4024-a2a2-55eb10dc76ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:57.044872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e0884-37a9-4024-a2a2-55eb10dc76ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:57.046909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e0884-37a9-4024-a2a2-55eb10dc76ab service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:57.080664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1109/2229] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:57.778503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809945-d63f-48f0-a1a9-42ba19d2f88a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:57.780091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a9809945-d63f-48f0-a1a9-42ba19d2f88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:57.807002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809945-d63f-48f0-a1a9-42ba19d2f88a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:57.824483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:57.825440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:57.825910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:57.826329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] PATH_INFO: `/v3/projects/fc4ee3fd0f8f42e491e3c47cb6c27899` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:57.827930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4ee3fd0f8f42e491e3c47cb6c27899)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:57.830389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:57.922406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4ee3fd0f8f42e491e3c47cb6c27899', 'request_id': 'req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:57.928980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4ee3fd0f8f42e491e3c47cb6c27899', 'request_id': 'req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:57.939444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4ee3fd0f8f42e491e3c47cb6c27899', 'request_id': 'req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:57.955237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] The token cache is being invalidate because project fc4ee3fd0f8f42e491e3c47cb6c27899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:20:57.955518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:57.956339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3647cbaf-c13c-40e9-acfe-52e45cfb2f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:20:57.958186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1121/2230] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:20:57 2022] DELETE /identity/v3/projects/fc4ee3fd0f8f42e491e3c47cb6c27899 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:20:57.963172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50134-dba6-4c1c-90f6-7b41ddf27c8d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:57.963694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50134-dba6-4c1c-90f6-7b41ddf27c8d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:57.963806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de50134-dba6-4c1c-90f6-7b41ddf27c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.007309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4de50134-dba6-4c1c-90f6-7b41ddf27c8d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:58.008535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4de50134-dba6-4c1c-90f6-7b41ddf27c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:58.094178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1110/2231] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:58.171332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bb782-bcf7-4823-b915-301b13f75dca None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:58.171614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bb782-bcf7-4823-b915-301b13f75dca None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:58.173024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bb782-bcf7-4823-b915-301b13f75dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.202354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-5d8bb782-bcf7-4823-b915-301b13f75dca None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:58.203634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5d8bb782-bcf7-4823-b915-301b13f75dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:58.213912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1122/2232] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:58.217742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de50134-dba6-4c1c-90f6-7b41ddf27c8d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:58.231900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de50134-dba6-4c1c-90f6-7b41ddf27c8d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:58.238741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:58.239658 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:58.240119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:58.240602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.241305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:20:58.241901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:58.244205 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:58.261063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1111/2233] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:20:58.265281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bb782-bcf7-4823-b915-301b13f75dca None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:58.277238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bb782-bcf7-4823-b915-301b13f75dca None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:58.288365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae26ea4-5163-4bca-a79d-85cee4f9a9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:58.289495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae26ea4-5163-4bca-a79d-85cee4f9a9fe admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:58.289956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae26ea4-5163-4bca-a79d-85cee4f9a9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:58.290428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae26ea4-5163-4bca-a79d-85cee4f9a9fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.291248 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae26ea4-5163-4bca-a79d-85cee4f9a9fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:58.293399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae26ea4-5163-4bca-a79d-85cee4f9a9fe admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:58.331398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1123/2234] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:20:58 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:58.335626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:58.352023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc00547-91af-4135-a75f-821ff0031c0b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:58.457403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf602f-63a5-489e-931e-a11730e1e279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:58.458771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf602f-63a5-489e-931e-a11730e1e279 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:58.459367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf602f-63a5-489e-931e-a11730e1e279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:58.460162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf602f-63a5-489e-931e-a11730e1e279 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.462568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf602f-63a5-489e-931e-a11730e1e279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:58.466115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf602f-63a5-489e-931e-a11730e1e279 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:58.506779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1112/2235] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:20:58 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:20:58.511256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae26ea4-5163-4bca-a79d-85cee4f9a9fe admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:58.526084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae26ea4-5163-4bca-a79d-85cee4f9a9fe admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:58.541620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c7b9d-764d-4062-a15f-546b270a5553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:58.542571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c7b9d-764d-4062-a15f-546b270a5553 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:58.542864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c7b9d-764d-4062-a15f-546b270a5553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:58.543136 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c7b9d-764d-4062-a15f-546b270a5553 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.543816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c7b9d-764d-4062-a15f-546b270a5553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:58.546058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c7b9d-764d-4062-a15f-546b270a5553 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:58.601995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1124/2236] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:20:58.606172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf602f-63a5-489e-931e-a11730e1e279 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:58.624111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf602f-63a5-489e-931e-a11730e1e279 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:58.636533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ccf4d-2a27-4a8e-b4e2-bbc600a0bcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:58.637438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ccf4d-2a27-4a8e-b4e2-bbc600a0bcfd admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:58.639007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ccf4d-2a27-4a8e-b4e2-bbc600a0bcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:58.639350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ccf4d-2a27-4a8e-b4e2-bbc600a0bcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.641853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ccf4d-2a27-4a8e-b4e2-bbc600a0bcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:58.646525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ccf4d-2a27-4a8e-b4e2-bbc600a0bcfd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:58.691961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1113/2237] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:20:58.693658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c7b9d-764d-4062-a15f-546b270a5553 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:58.711829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c7b9d-764d-4062-a15f-546b270a5553 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:58.724425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:58.725230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:58.725703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:58.726046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] PATH_INFO: `/v3/projects/d9f01988234948f18e7182e829cbcf0b/users/c5efa16ebc44403b8382ecc784e1eddf/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.773557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f01988234948f18e7182e829cbcf0b, user_id=c5efa16ebc44403b8382ecc784e1eddf, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:58.778429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:58.813686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1125/2238] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:20:58 2022] PUT /identity/v3/projects/d9f01988234948f18e7182e829cbcf0b/users/c5efa16ebc44403b8382ecc784e1eddf/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:20:58.823614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5592-05cc-4e71-b526-b549b24dd78f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:58.823845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5592-05cc-4e71-b526-b549b24dd78f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:58.824162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5592-05cc-4e71-b526-b549b24dd78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:58.876131 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7afd5592-05cc-4e71-b526-b549b24dd78f None None] MFA Rules not processed for user `c5efa16ebc44403b8382ecc784e1eddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:58.877658 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7afd5592-05cc-4e71-b526-b549b24dd78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:58.895846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:59.012072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1114/2239] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:20:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:20:59.025489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:59.027229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:59.085898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:59.100461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3193b5-fed5-4af9-b93c-29e2b2223f16 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:59.106292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae1f33-306c-4ee9-bf5e-44b31fb31a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:59.107396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1f33-306c-4ee9-bf5e-44b31fb31a09 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:59.107948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1f33-306c-4ee9-bf5e-44b31fb31a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:59.108406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae1f33-306c-4ee9-bf5e-44b31fb31a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:59.121057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae1f33-306c-4ee9-bf5e-44b31fb31a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:59.123280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae1f33-306c-4ee9-bf5e-44b31fb31a09 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:59.138861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1126/2240] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:59.787663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd5592-05cc-4e71-b526-b549b24dd78f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:59.789775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7afd5592-05cc-4e71-b526-b549b24dd78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:59.949083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd5592-05cc-4e71-b526-b549b24dd78f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:59.955299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:59.956357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:59.956955 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:59.957383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] PATH_INFO: `/v3/users/b7c7e84a2809475cb7fbfaf905c2ad5c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:59.958742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c7e84a2809475cb7fbfaf905c2ad5c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:59.961287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:00.028125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c7e84a2809475cb7fbfaf905c2ad5c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.044946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c7e84a2809475cb7fbfaf905c2ad5c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.045692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c7e84a2809475cb7fbfaf905c2ad5c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.046308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c7e84a2809475cb7fbfaf905c2ad5c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.056777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a5889435-1386-47db-b53b-ab87f65e28c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c7e84a2809475cb7fbfaf905c2ad5c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.072832 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1115/2241] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:20:59 2022] DELETE /identity/v3/users/b7c7e84a2809475cb7fbfaf905c2ad5c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:00.455046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae1f33-306c-4ee9-bf5e-44b31fb31a09 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:00.472647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae1f33-306c-4ee9-bf5e-44b31fb31a09 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:00.563987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:00.564712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:00.564987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:00.565298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] PATH_INFO: `/v3/projects/47b4c76757374438b8d4b0b7148be568` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:00.566501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47b4c76757374438b8d4b0b7148be568)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:00.568508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:00.628927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47b4c76757374438b8d4b0b7148be568', 'request_id': 'req-0d203052-eba1-4206-9068-ca766b3765e2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.635427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47b4c76757374438b8d4b0b7148be568', 'request_id': 'req-0d203052-eba1-4206-9068-ca766b3765e2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.644939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47b4c76757374438b8d4b0b7148be568', 'request_id': 'req-0d203052-eba1-4206-9068-ca766b3765e2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.663397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] The token cache is being invalidate because project 47b4c76757374438b8d4b0b7148be568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:21:00.664101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.665793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0d203052-eba1-4206-9068-ca766b3765e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:00.670990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1127/2242] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:21:00 2022] DELETE /identity/v3/projects/47b4c76757374438b8d4b0b7148be568 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:00.680313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c5244-650d-4f2d-bc4f-b5487e37b06f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:00.680829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c5244-650d-4f2d-bc4f-b5487e37b06f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:00.681305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c5244-650d-4f2d-bc4f-b5487e37b06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:00.724893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0b4c5244-650d-4f2d-bc4f-b5487e37b06f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:00.727632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0b4c5244-650d-4f2d-bc4f-b5487e37b06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:00.789338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:00.837608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1116/2243] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:00.846589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdfd09-70d5-40fc-884e-6e55292d99cf None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:00.847369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdfd09-70d5-40fc-884e-6e55292d99cf None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:00.847791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdfd09-70d5-40fc-884e-6e55292d99cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:00.872327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-eacdfd09-70d5-40fc-884e-6e55292d99cf None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:00.873739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-eacdfd09-70d5-40fc-884e-6e55292d99cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:00.883861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1128/2244] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:00.887041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c5244-650d-4f2d-bc4f-b5487e37b06f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:00.900482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c5244-650d-4f2d-bc4f-b5487e37b06f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:00.908506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:00.909308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:00.909682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:00.910056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:00.911312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:21:00.912142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:00.916777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:00.956346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1117/2245] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:21:00.960074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdfd09-70d5-40fc-884e-6e55292d99cf None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:00.987610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdfd09-70d5-40fc-884e-6e55292d99cf None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:01.008450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2559989-22e5-4505-902f-079f57838146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:01.009529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2559989-22e5-4505-902f-079f57838146 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:01.010003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2559989-22e5-4505-902f-079f57838146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:01.010467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2559989-22e5-4505-902f-079f57838146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:01.011847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2559989-22e5-4505-902f-079f57838146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:01.016370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2559989-22e5-4505-902f-079f57838146 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:01.083897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1129/2246] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:21:00 2022] POST /identity/v3/projects => generated 397 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:01.088760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:01.111979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81faf8d-7155-47d0-a3f2-ec0718dce36c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:01.199515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd070d-baa0-486e-aaf7-e42524438ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:01.200676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd070d-baa0-486e-aaf7-e42524438ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:01.201135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd070d-baa0-486e-aaf7-e42524438ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:01.201649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd070d-baa0-486e-aaf7-e42524438ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:01.202302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd070d-baa0-486e-aaf7-e42524438ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:01.205910 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd070d-baa0-486e-aaf7-e42524438ea9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:01.232620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1118/2247] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:21:01 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:01.237179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2559989-22e5-4505-902f-079f57838146 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:01.249035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2559989-22e5-4505-902f-079f57838146 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:01.254086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e9e60-d588-43e4-b5b6-00ec4d5d956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:01.254805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e9e60-d588-43e4-b5b6-00ec4d5d956e admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:01.255167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e9e60-d588-43e4-b5b6-00ec4d5d956e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:01.255669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e9e60-d588-43e4-b5b6-00ec4d5d956e admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:01.256351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e9e60-d588-43e4-b5b6-00ec4d5d956e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:01.258216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e9e60-d588-43e4-b5b6-00ec4d5d956e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:01.271252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1130/2248] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:21:01.275088 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd070d-baa0-486e-aaf7-e42524438ea9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:01.289734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd070d-baa0-486e-aaf7-e42524438ea9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:01.295470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751371b7-a9d4-4cb0-92ab-d08dd95e4337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:01.296257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751371b7-a9d4-4cb0-92ab-d08dd95e4337 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:01.296257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751371b7-a9d4-4cb0-92ab-d08dd95e4337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:01.297606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751371b7-a9d4-4cb0-92ab-d08dd95e4337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:01.297606 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751371b7-a9d4-4cb0-92ab-d08dd95e4337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:01.298877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751371b7-a9d4-4cb0-92ab-d08dd95e4337 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:01.327002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e9e60-d588-43e4-b5b6-00ec4d5d956e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:01.327439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1119/2249] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:01.340806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e9e60-d588-43e4-b5b6-00ec4d5d956e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:01.347154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:01.347884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:01.348177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:01.348502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] PATH_INFO: `/v3/projects/7a9d607b63e44e18b829a48077657902/users/67c3ad0e0f6c4e51bf4bbddf54c7a824/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:01.366911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9d607b63e44e18b829a48077657902, user_id=67c3ad0e0f6c4e51bf4bbddf54c7a824, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:01.369665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:01.402229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1131/2250] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:21:01 2022] PUT /identity/v3/projects/7a9d607b63e44e18b829a48077657902/users/67c3ad0e0f6c4e51bf4bbddf54c7a824/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:01.402523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d9dd5-a5d0-496f-b850-2ab4f9910388 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:01.402980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d9dd5-a5d0-496f-b850-2ab4f9910388 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:01.403436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d9dd5-a5d0-496f-b850-2ab4f9910388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:01.475354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-342d9dd5-a5d0-496f-b850-2ab4f9910388 None None] MFA Rules not processed for user `67c3ad0e0f6c4e51bf4bbddf54c7a824`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:01.477520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-342d9dd5-a5d0-496f-b850-2ab4f9910388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:01.628272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1120/2251] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:01.635482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:01.637386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:01.652142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:01.738777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf8535-a4cd-422c-8b2c-99c9f4e4b9a2 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:01.748186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:01.750121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:01.750987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:01.751427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:01.761659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:01.763372 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:01.778291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1132/2252] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:01 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:03.017763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346584d-2e01-4cb5-a991-2f7461e1190c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:03.018227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346584d-2e01-4cb5-a991-2f7461e1190c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:03.018312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346584d-2e01-4cb5-a991-2f7461e1190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:03.040040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f346584d-2e01-4cb5-a991-2f7461e1190c None None] MFA Rules not processed for user `c5efa16ebc44403b8382ecc784e1eddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:03.041310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f346584d-2e01-4cb5-a991-2f7461e1190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:03.090253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:03.209155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1121/2253] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:10.197626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:10.199726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:10.268342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af148043-ee81-4037-883d-6ffe7e857a5c service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:10.273056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:10.273692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:10.273974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:10.274246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] PATH_INFO: `/v3/users/668428c74f98417987ee6c54ee9181d2` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:10.275380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=668428c74f98417987ee6c54ee9181d2)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:10.277367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:10.327335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668428c74f98417987ee6c54ee9181d2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.335562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668428c74f98417987ee6c54ee9181d2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.336322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668428c74f98417987ee6c54ee9181d2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.349400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668428c74f98417987ee6c54ee9181d2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.350182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668428c74f98417987ee6c54ee9181d2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.363664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1133/2254] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:21:10 2022] DELETE /identity/v3/users/668428c74f98417987ee6c54ee9181d2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:10.368288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298c3da-b788-4a18-b778-6c0524cdab66 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:10.368888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298c3da-b788-4a18-b778-6c0524cdab66 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:10.368888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298c3da-b788-4a18-b778-6c0524cdab66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:10.392805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3298c3da-b788-4a18-b778-6c0524cdab66 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:10.394324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3298c3da-b788-4a18-b778-6c0524cdab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:10.404056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:10.405183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:10.496867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1122/2255] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:10.518977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:10.552263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938ac33-186e-422b-a26b-15454d2f98d4 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:10.558185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:10.559037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:10.559364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:10.559948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] PATH_INFO: `/v3/users/67c3ad0e0f6c4e51bf4bbddf54c7a824` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:10.561893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298c3da-b788-4a18-b778-6c0524cdab66 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:10.568172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3298c3da-b788-4a18-b778-6c0524cdab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:10.569497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c3ad0e0f6c4e51bf4bbddf54c7a824)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:10.571464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:10.629358 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c3ad0e0f6c4e51bf4bbddf54c7a824'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.638394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c3ad0e0f6c4e51bf4bbddf54c7a824'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.639142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c3ad0e0f6c4e51bf4bbddf54c7a824'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.641881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298c3da-b788-4a18-b778-6c0524cdab66 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:10.647162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:10.648015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:10.648787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:10.649287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:10.651619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:10.654202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c3ad0e0f6c4e51bf4bbddf54c7a824'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.655007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-bbeaab29-11ef-4784-9958-41566565a9b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c3ad0e0f6c4e51bf4bbddf54c7a824'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:10.666182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:10.667954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:10.680086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1134/2256] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:21:10 2022] DELETE /identity/v3/users/67c3ad0e0f6c4e51bf4bbddf54c7a824 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:10.680978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1123/2257] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:21:10.684655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ec18b-3139-4768-9bdd-8e8510bc37cb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:10.685198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ec18b-3139-4768-9bdd-8e8510bc37cb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:10.686598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ec18b-3139-4768-9bdd-8e8510bc37cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:10.714261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6f0ec18b-3139-4768-9bdd-8e8510bc37cb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:10.716325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6f0ec18b-3139-4768-9bdd-8e8510bc37cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:10.814166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1135/2258] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:21:10.854519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:10.856102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:10.871895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce95407-3a04-4401-ad4f-2cb5ba9a44ae service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:10.936509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:10.936751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:10.937806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:10.939100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:10.951618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:10.953288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:10.970510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1124/2259] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:11.442650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ec18b-3139-4768-9bdd-8e8510bc37cb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.444702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6f0ec18b-3139-4768-9bdd-8e8510bc37cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.450893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.453276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.472578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ec18b-3139-4768-9bdd-8e8510bc37cb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:11.477113 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b018d-526d-40a5-a341-a59a9f4c5314 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:11.488607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:11.489443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:11.490383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.490818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.490956 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] PATH_INFO: `/v3/projects/ffaeee7ebbe34372a16f7a8b7b309e03` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.491489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.491763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.492101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] PATH_INFO: `/v3/projects/7a9d607b63e44e18b829a48077657902` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.493842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffaeee7ebbe34372a16f7a8b7b309e03)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:11.495639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9d607b63e44e18b829a48077657902)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:11.500088 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:11.501006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:11.533203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:11.563334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9d607b63e44e18b829a48077657902', 'request_id': 'req-4fe86249-7389-4418-9120-c4e07f1cb62d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.565033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:11.567924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffaeee7ebbe34372a16f7a8b7b309e03', 'request_id': 'req-bfbb5f0e-65c5-495d-bb81-baad1988ddad'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.571928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9d607b63e44e18b829a48077657902', 'request_id': 'req-4fe86249-7389-4418-9120-c4e07f1cb62d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.578368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffaeee7ebbe34372a16f7a8b7b309e03', 'request_id': 'req-bfbb5f0e-65c5-495d-bb81-baad1988ddad'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.581053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9d607b63e44e18b829a48077657902', 'request_id': 'req-4fe86249-7389-4418-9120-c4e07f1cb62d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.586528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffaeee7ebbe34372a16f7a8b7b309e03', 'request_id': 'req-bfbb5f0e-65c5-495d-bb81-baad1988ddad'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.597966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] The token cache is being invalidate because project 7a9d607b63e44e18b829a48077657902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:21:11.598385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.599159 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe86249-7389-4418-9120-c4e07f1cb62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.599483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] The token cache is being invalidate because project ffaeee7ebbe34372a16f7a8b7b309e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:21:11.599764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.600411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bfbb5f0e-65c5-495d-bb81-baad1988ddad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:11.601126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1136/2260] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:21:11 2022] DELETE /identity/v3/projects/7a9d607b63e44e18b829a48077657902 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:11.602024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4916f-27cb-4189-8672-29b7a1fd2523 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.602308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4916f-27cb-4189-8672-29b7a1fd2523 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.603042 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4916f-27cb-4189-8672-29b7a1fd2523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.609105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1125/2261] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:21:11 2022] DELETE /identity/v3/projects/ffaeee7ebbe34372a16f7a8b7b309e03 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:11.610592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121b9bf-02db-46f7-a4e1-adfd933d0071 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.611164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121b9bf-02db-46f7-a4e1-adfd933d0071 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.611318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121b9bf-02db-46f7-a4e1-adfd933d0071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.625128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-05a4916f-27cb-4189-8672-29b7a1fd2523 None None] MFA Rules not processed for user `fd0f7f51482040d7a89df3f7502c2638`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:11.626377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-05a4916f-27cb-4189-8672-29b7a1fd2523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.630984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c121b9bf-02db-46f7-a4e1-adfd933d0071 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:11.632163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c121b9bf-02db-46f7-a4e1-adfd933d0071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.685624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1137/2262] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:11.686910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafff1cb-a43b-4f5e-9e82-f57dd2648156 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.687357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafff1cb-a43b-4f5e-9e82-f57dd2648156 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.687635 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafff1cb-a43b-4f5e-9e82-f57dd2648156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.699047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1126/2263] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:11.726524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-cafff1cb-a43b-4f5e-9e82-f57dd2648156 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:11.727876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cafff1cb-a43b-4f5e-9e82-f57dd2648156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.738324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1138/2264] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:11.746398 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947608c5-a26e-48ed-bdb7-a150fe297f82 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.746682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947608c5-a26e-48ed-bdb7-a150fe297f82 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.746971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947608c5-a26e-48ed-bdb7-a150fe297f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.770050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e6b7e-9d03-4956-964d-e2e62cd7434a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.770338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e6b7e-9d03-4956-964d-e2e62cd7434a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.770611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e6b7e-9d03-4956-964d-e2e62cd7434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.775244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-947608c5-a26e-48ed-bdb7-a150fe297f82 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:11.776466 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-947608c5-a26e-48ed-bdb7-a150fe297f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.791587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1127/2265] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:11.792555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947608c5-a26e-48ed-bdb7-a150fe297f82 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.800085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f79e6b7e-9d03-4956-964d-e2e62cd7434a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:11.801527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f79e6b7e-9d03-4956-964d-e2e62cd7434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.804322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947608c5-a26e-48ed-bdb7-a150fe297f82 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:11.809121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:11.810085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.810379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.810656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.811332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1139/2266] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:11.811734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:21:11.812754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:11.816405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e6b7e-9d03-4956-964d-e2e62cd7434a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.816586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:11.825903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e6b7e-9d03-4956-964d-e2e62cd7434a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:11.832316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:11.833053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.833392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.834137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.834442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:21:11.834803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:11.836600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:11.839067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1128/2267] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:21:11.840061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.851166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1140/2268] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:21:11.851562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8c2be-c855-406d-8e5d-3a2c75a99458 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:11.855531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.860048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a602c7-9973-4b5b-a72e-c8199af59cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:11.861117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a602c7-9973-4b5b-a72e-c8199af59cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.861117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a602c7-9973-4b5b-a72e-c8199af59cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.861442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a602c7-9973-4b5b-a72e-c8199af59cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.862149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a602c7-9973-4b5b-a72e-c8199af59cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:11.864311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a602c7-9973-4b5b-a72e-c8199af59cc5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:11.866228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6702589-9573-46c8-87f7-ba6d4e88474e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:11.879908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37546f-bf8c-47eb-8936-27cb326bb272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:11.880520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37546f-bf8c-47eb-8936-27cb326bb272 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.880828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37546f-bf8c-47eb-8936-27cb326bb272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.881098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37546f-bf8c-47eb-8936-27cb326bb272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.881733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37546f-bf8c-47eb-8936-27cb326bb272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:11.883717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37546f-bf8c-47eb-8936-27cb326bb272 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:11.904960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1129/2269] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:11.909925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a602c7-9973-4b5b-a72e-c8199af59cc5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.926262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1141/2270] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:21:11.927656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37546f-bf8c-47eb-8936-27cb326bb272 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.983198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a602c7-9973-4b5b-a72e-c8199af59cc5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:11.991052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:12.007791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37546f-bf8c-47eb-8936-27cb326bb272 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.021502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89423551-0e8b-4e85-babb-577376da7d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.022359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89423551-0e8b-4e85-babb-577376da7d71 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.022635 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89423551-0e8b-4e85-babb-577376da7d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.023151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89423551-0e8b-4e85-babb-577376da7d71 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.024284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89423551-0e8b-4e85-babb-577376da7d71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.027899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89423551-0e8b-4e85-babb-577376da7d71 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.070025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f61ae6-41d4-45e2-8fa5-ee68cba4782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.070871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61ae6-41d4-45e2-8fa5-ee68cba4782c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.071155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61ae6-41d4-45e2-8fa5-ee68cba4782c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.071433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61ae6-41d4-45e2-8fa5-ee68cba4782c admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.072100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f61ae6-41d4-45e2-8fa5-ee68cba4782c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.072813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1142/2271] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:12.074244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f61ae6-41d4-45e2-8fa5-ee68cba4782c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.077719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89423551-0e8b-4e85-babb-577376da7d71 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:12.090106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89423551-0e8b-4e85-babb-577376da7d71 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.096885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4ff7d-5b42-4046-861c-b32de7d3fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.097721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ff7d-5b42-4046-861c-b32de7d3fb5d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.098164 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ff7d-5b42-4046-861c-b32de7d3fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.098604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ff7d-5b42-4046-861c-b32de7d3fb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.099262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4ff7d-5b42-4046-861c-b32de7d3fb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.101237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4ff7d-5b42-4046-861c-b32de7d3fb5d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.108267 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1130/2272] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:12.110077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f61ae6-41d4-45e2-8fa5-ee68cba4782c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:12.118146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1143/2273] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:21:12.119168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4ff7d-5b42-4046-861c-b32de7d3fb5d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:12.130194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4ff7d-5b42-4046-861c-b32de7d3fb5d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.133225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f61ae6-41d4-45e2-8fa5-ee68cba4782c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.135841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6c881-92b2-4c2b-aa5d-3f0edb07c58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.136623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6c881-92b2-4c2b-aa5d-3f0edb07c58a admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.136994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6c881-92b2-4c2b-aa5d-3f0edb07c58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.137336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6c881-92b2-4c2b-aa5d-3f0edb07c58a admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.138052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6c881-92b2-4c2b-aa5d-3f0edb07c58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.139771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6c881-92b2-4c2b-aa5d-3f0edb07c58a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.142887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e163ed-3c42-4ef4-a338-ef46eda74a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.143720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e163ed-3c42-4ef4-a338-ef46eda74a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.144196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e163ed-3c42-4ef4-a338-ef46eda74a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.144653 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e163ed-3c42-4ef4-a338-ef46eda74a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.146452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e163ed-3c42-4ef4-a338-ef46eda74a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.148303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e163ed-3c42-4ef4-a338-ef46eda74a50 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.157170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1144/2274] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:21:12.158087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6c881-92b2-4c2b-aa5d-3f0edb07c58a admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:12.164539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1131/2275] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:12.164671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:12.167762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e163ed-3c42-4ef4-a338-ef46eda74a50 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:12.171186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6c881-92b2-4c2b-aa5d-3f0edb07c58a admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.179325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5cd40-ec6a-4e3a-bd07-a3116c7bf63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.180248 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5cd40-ec6a-4e3a-bd07-a3116c7bf63b admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.180534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5cd40-ec6a-4e3a-bd07-a3116c7bf63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.180771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e163ed-3c42-4ef4-a338-ef46eda74a50 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.180988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5cd40-ec6a-4e3a-bd07-a3116c7bf63b admin admin] PATH_INFO: `/v3/projects/a16d7c3be2644b64b9d43e77d9e45804/users/3e8d388698c6493c83f83059727a6188/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.186220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed61f8-46aa-4f44-ac8b-742e968c275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.186932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed61f8-46aa-4f44-ac8b-742e968c275d admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.187913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed61f8-46aa-4f44-ac8b-742e968c275d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.188188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed61f8-46aa-4f44-ac8b-742e968c275d admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.189074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed61f8-46aa-4f44-ac8b-742e968c275d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.190708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed61f8-46aa-4f44-ac8b-742e968c275d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.198800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5cd40-ec6a-4e3a-bd07-a3116c7bf63b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16d7c3be2644b64b9d43e77d9e45804, user_id=3e8d388698c6493c83f83059727a6188, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.201782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5cd40-ec6a-4e3a-bd07-a3116c7bf63b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.204629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1132/2276] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:12.207705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed61f8-46aa-4f44-ac8b-742e968c275d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:12.222112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed61f8-46aa-4f44-ac8b-742e968c275d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.222385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1145/2277] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:21:12 2022] PUT /identity/v3/projects/a16d7c3be2644b64b9d43e77d9e45804/users/3e8d388698c6493c83f83059727a6188/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:12.230385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e964b8-e500-463e-bee3-5087215052fc None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.230699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e964b8-e500-463e-bee3-5087215052fc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.230944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e964b8-e500-463e-bee3-5087215052fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.256538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d0e964b8-e500-463e-bee3-5087215052fc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:12.257849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d0e964b8-e500-463e-bee3-5087215052fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:12.295576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d0aa0-a02d-4239-b1ea-4f7d50a648ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.296846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0aa0-a02d-4239-b1ea-4f7d50a648ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.297144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0aa0-a02d-4239-b1ea-4f7d50a648ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.297396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0aa0-a02d-4239-b1ea-4f7d50a648ff admin admin] PATH_INFO: `/v3/projects/3b530b6cef6943ee93e561b05d1f846d/users/1e04230bb5b34b27aad5c2a46230a5cc/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.315197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d0aa0-a02d-4239-b1ea-4f7d50a648ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b530b6cef6943ee93e561b05d1f846d, user_id=1e04230bb5b34b27aad5c2a46230a5cc, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.317931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d0aa0-a02d-4239-b1ea-4f7d50a648ff admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.339337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1133/2278] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:21:12 2022] PUT /identity/v3/projects/3b530b6cef6943ee93e561b05d1f846d/users/1e04230bb5b34b27aad5c2a46230a5cc/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:12.344782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909981-73ca-4180-9383-0bc9feb6a42f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.345129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909981-73ca-4180-9383-0bc9feb6a42f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.345411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909981-73ca-4180-9383-0bc9feb6a42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.365092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1146/2279] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:12.366450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cd909981-73ca-4180-9383-0bc9feb6a42f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:12.367769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cd909981-73ca-4180-9383-0bc9feb6a42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:12.402687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e964b8-e500-463e-bee3-5087215052fc None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:12.404853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d0e964b8-e500-463e-bee3-5087215052fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:12.443136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:12.460372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1134/2280] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:12.474198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd909981-73ca-4180-9383-0bc9feb6a42f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:12.476170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cd909981-73ca-4180-9383-0bc9feb6a42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:12.481977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e964b8-e500-463e-bee3-5087215052fc None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.545591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd909981-73ca-4180-9383-0bc9feb6a42f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:12.550534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31c0b5-e8a2-4e6a-9341-7f72dd3a7339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.551160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c0b5-e8a2-4e6a-9341-7f72dd3a7339 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.551443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c0b5-e8a2-4e6a-9341-7f72dd3a7339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.551717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c0b5-e8a2-4e6a-9341-7f72dd3a7339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.560297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:12.560970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:12.561283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:12.561531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:12.563688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31c0b5-e8a2-4e6a-9341-7f72dd3a7339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.566030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31c0b5-e8a2-4e6a-9341-7f72dd3a7339 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.574243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:12.575767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:12.579761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1135/2281] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:21:12.588057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1147/2282] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:13.369413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f38fda-9e4d-4828-8f5d-cc2a12d37310 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:13.369704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f38fda-9e4d-4828-8f5d-cc2a12d37310 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:13.369995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f38fda-9e4d-4828-8f5d-cc2a12d37310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:13.498447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-13f38fda-9e4d-4828-8f5d-cc2a12d37310 None None] MFA Rules not processed for user `1e04230bb5b34b27aad5c2a46230a5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:13.499923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-13f38fda-9e4d-4828-8f5d-cc2a12d37310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:13.650441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1136/2283] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:13.673432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:13.676329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:13.736011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:13.769914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fe30c-3bb1-43ba-8118-446e8e684080 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:13.774500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56451b8f-5b2c-43c4-99aa-7809491c5ccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:13.775855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451b8f-5b2c-43c4-99aa-7809491c5ccf service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:13.776460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451b8f-5b2c-43c4-99aa-7809491c5ccf service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:13.776941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451b8f-5b2c-43c4-99aa-7809491c5ccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:13.795454 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56451b8f-5b2c-43c4-99aa-7809491c5ccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:13.799378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56451b8f-5b2c-43c4-99aa-7809491c5ccf service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:13.812990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1148/2284] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:15.652283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bdd46-121f-44a9-8e4b-1d3155574cd7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:15.653062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bdd46-121f-44a9-8e4b-1d3155574cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:15.653062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327bdd46-121f-44a9-8e4b-1d3155574cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:15.688943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-327bdd46-121f-44a9-8e4b-1d3155574cd7 None None] MFA Rules not processed for user `1e04230bb5b34b27aad5c2a46230a5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:15.690947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-327bdd46-121f-44a9-8e4b-1d3155574cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:15.713659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1137/2285] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:16.721687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36cbdd-2eec-44af-abd6-47f404961d4a None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:16.722675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36cbdd-2eec-44af-abd6-47f404961d4a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:16.723048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36cbdd-2eec-44af-abd6-47f404961d4a None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:16.724493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1149/2286] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:21:16.730809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea3b5-07a9-4a2f-b9a6-a6a1c1a87f15 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:16.731301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea3b5-07a9-4a2f-b9a6-a6a1c1a87f15 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:16.731394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea3b5-07a9-4a2f-b9a6-a6a1c1a87f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:16.752085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f52ea3b5-07a9-4a2f-b9a6-a6a1c1a87f15 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:16.753538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f52ea3b5-07a9-4a2f-b9a6-a6a1c1a87f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:16.834151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1138/2287] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:21:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:16.846714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36cbdd-2eec-44af-abd6-47f404961d4a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:16.848853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4b36cbdd-2eec-44af-abd6-47f404961d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:16.918290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36cbdd-2eec-44af-abd6-47f404961d4a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:16.921933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2ecbe-4d46-4b7f-9b3b-4f8193bd10b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6c90a2b95c498494a73b43e7138905', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:16.922802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2ecbe-4d46-4b7f-9b3b-4f8193bd10b5 service glance] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:16.923177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2ecbe-4d46-4b7f-9b3b-4f8193bd10b5 service glance] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:16.923543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2ecbe-4d46-4b7f-9b3b-4f8193bd10b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:16.934902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2ecbe-4d46-4b7f-9b3b-4f8193bd10b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:16.936495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2ecbe-4d46-4b7f-9b3b-4f8193bd10b5 service glance] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:16.960058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1150/2288] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:17.079138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e5fcb-ef42-4a7a-bef7-0bade4292eaf None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.079425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e5fcb-ef42-4a7a-bef7-0bade4292eaf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.080743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e5fcb-ef42-4a7a-bef7-0bade4292eaf None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.084506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1139/2289] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:21:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:21:17.089471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2b482-e83a-4308-8a11-6f398b8141a5 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.089858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2b482-e83a-4308-8a11-6f398b8141a5 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.090282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2b482-e83a-4308-8a11-6f398b8141a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.133479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-74b2b482-e83a-4308-8a11-6f398b8141a5 None None] MFA Rules not processed for user `5f1d5c29df30465ead9834b63c3e93bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:17.137083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-74b2b482-e83a-4308-8a11-6f398b8141a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:17.167698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1151/2290] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:17.221255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e5fcb-ef42-4a7a-bef7-0bade4292eaf None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:17.240730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e5fcb-ef42-4a7a-bef7-0bade4292eaf None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:17.249210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813479f6-bb14-46da-a7d3-a542da593c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:17.249874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813479f6-bb14-46da-a7d3-a542da593c2b service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.257151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813479f6-bb14-46da-a7d3-a542da593c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.257590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813479f6-bb14-46da-a7d3-a542da593c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.281372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813479f6-bb14-46da-a7d3-a542da593c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:17.282847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813479f6-bb14-46da-a7d3-a542da593c2b service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:17.303264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1140/2291] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:21:17.645922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c263b3-4d77-4ab5-89a6-82aaa1905a63 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.646216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c263b3-4d77-4ab5-89a6-82aaa1905a63 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.646493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c263b3-4d77-4ab5-89a6-82aaa1905a63 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.647875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1152/2292] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:21:17.659787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717c64-4f77-4b38-ac2c-f124f2fcda53 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.660074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717c64-4f77-4b38-ac2c-f124f2fcda53 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.660367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717c64-4f77-4b38-ac2c-f124f2fcda53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.674913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:17.705775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-46717c64-4f77-4b38-ac2c-f124f2fcda53 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:17.707069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-46717c64-4f77-4b38-ac2c-f124f2fcda53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:17.726539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1141/2293] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:17.813607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c263b3-4d77-4ab5-89a6-82aaa1905a63 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:17.841398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c263b3-4d77-4ab5-89a6-82aaa1905a63 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:17.866182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ab3b8-990d-4338-b6a7-6301d840d9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:17.867715 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab3b8-990d-4338-b6a7-6301d840d9aa service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:17.868078 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab3b8-990d-4338-b6a7-6301d840d9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:17.868352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab3b8-990d-4338-b6a7-6301d840d9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:17.900994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ab3b8-990d-4338-b6a7-6301d840d9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:17.903053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ab3b8-990d-4338-b6a7-6301d840d9aa service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:17.933107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1153/2294] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:20.850405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4ee0-6e65-4abb-bd84-9d198e359cce None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:20.850865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4ee0-6e65-4abb-bd84-9d198e359cce None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:20.851141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4ee0-6e65-4abb-bd84-9d198e359cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:20.881994 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-101d4ee0-6e65-4abb-bd84-9d198e359cce None None] MFA Rules not processed for user `3e8d388698c6493c83f83059727a6188`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:20.884358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-101d4ee0-6e65-4abb-bd84-9d198e359cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:21.041143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1142/2295] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:21.042932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ab3b8-990d-4338-b6a7-6301d840d9aa service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:21.057233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ab3b8-990d-4338-b6a7-6301d840d9aa service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:21.065894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4ce21-673d-4b4c-ad2f-78caebe4f109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:21.066732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ce21-673d-4b4c-ad2f-78caebe4f109 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:21.067246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ce21-673d-4b4c-ad2f-78caebe4f109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:21.067745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ce21-673d-4b4c-ad2f-78caebe4f109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:21.068893 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4ce21-673d-4b4c-ad2f-78caebe4f109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:21.071947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4ce21-673d-4b4c-ad2f-78caebe4f109 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:21.119928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d4ee0-6e65-4abb-bd84-9d198e359cce None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:21.129609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1154/2296] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:21:21 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:21.138052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d4ee0-6e65-4abb-bd84-9d198e359cce None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:21.218581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3ada4-8cc1-4c56-ac69-dc691aa88ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:21.218852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3ada4-8cc1-4c56-ac69-dc691aa88ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:21.219059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3ada4-8cc1-4c56-ac69-dc691aa88ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:21.219334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3ada4-8cc1-4c56-ac69-dc691aa88ccb admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:21.220016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3ada4-8cc1-4c56-ac69-dc691aa88ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:21.222279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3ada4-8cc1-4c56-ac69-dc691aa88ccb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:21.249906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1143/2297] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:21:21 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:21.253692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4ce21-673d-4b4c-ad2f-78caebe4f109 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:21.267124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4ce21-673d-4b4c-ad2f-78caebe4f109 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:21.272208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a31de-3708-45e0-af50-3e47553de45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:21.272890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a31de-3708-45e0-af50-3e47553de45f admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:21.273170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a31de-3708-45e0-af50-3e47553de45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:21.273441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a31de-3708-45e0-af50-3e47553de45f admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:21.274258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a31de-3708-45e0-af50-3e47553de45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:21.286847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a31de-3708-45e0-af50-3e47553de45f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:21.309791 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1155/2298] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:21.310569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3ada4-8cc1-4c56-ac69-dc691aa88ccb admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:21.327096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3ada4-8cc1-4c56-ac69-dc691aa88ccb admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:21.350895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e64435-ba03-493a-bf7c-42a41ce027dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:21.351697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64435-ba03-493a-bf7c-42a41ce027dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:21.351986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64435-ba03-493a-bf7c-42a41ce027dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:21.352244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64435-ba03-493a-bf7c-42a41ce027dc admin admin] PATH_INFO: `/v3/projects/1e249e3d4fea41b6a97aa8f2e060bbd1/users/1b94bf5be4e040d5a9732c81ce1dd077/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:21.432513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e64435-ba03-493a-bf7c-42a41ce027dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e249e3d4fea41b6a97aa8f2e060bbd1, user_id=1b94bf5be4e040d5a9732c81ce1dd077, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:21.451992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e64435-ba03-493a-bf7c-42a41ce027dc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:21.509684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1144/2299] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:21:21 2022] PUT /identity/v3/projects/1e249e3d4fea41b6a97aa8f2e060bbd1/users/1b94bf5be4e040d5a9732c81ce1dd077/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:21:21.511528 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a31de-3708-45e0-af50-3e47553de45f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:21.536610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a31de-3708-45e0-af50-3e47553de45f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:21.750607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d38a-7039-4d29-8d39-a6b540a4a661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:21.751253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d38a-7039-4d29-8d39-a6b540a4a661 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:21.751531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d38a-7039-4d29-8d39-a6b540a4a661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:21.751901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d38a-7039-4d29-8d39-a6b540a4a661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:21.754014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374d38a-7039-4d29-8d39-a6b540a4a661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:21.756084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374d38a-7039-4d29-8d39-a6b540a4a661 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:21.785217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1156/2300] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:21.789626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e64435-ba03-493a-bf7c-42a41ce027dc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:21.803149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e64435-ba03-493a-bf7c-42a41ce027dc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:21.845969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad65db-2d70-4ea0-a212-5c7d89aa75bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:21.846534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad65db-2d70-4ea0-a212-5c7d89aa75bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:21.846952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad65db-2d70-4ea0-a212-5c7d89aa75bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:21.847341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad65db-2d70-4ea0-a212-5c7d89aa75bd admin admin] PATH_INFO: `/v3/projects/1e249e3d4fea41b6a97aa8f2e060bbd1/users/1b94bf5be4e040d5a9732c81ce1dd077/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:21.850273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad65db-2d70-4ea0-a212-5c7d89aa75bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e249e3d4fea41b6a97aa8f2e060bbd1, user_id=1b94bf5be4e040d5a9732c81ce1dd077, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:21.854874 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad65db-2d70-4ea0-a212-5c7d89aa75bd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:21.871166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1145/2301] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:21:21 2022] PUT /identity/v3/projects/1e249e3d4fea41b6a97aa8f2e060bbd1/users/1b94bf5be4e040d5a9732c81ce1dd077/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:30.156980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51693a-4630-456d-9535-40beb91c07e0 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:30.157480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51693a-4630-456d-9535-40beb91c07e0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:30.157596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51693a-4630-456d-9535-40beb91c07e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:30.189146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fd51693a-4630-456d-9535-40beb91c07e0 None None] MFA Rules not processed for user `1b94bf5be4e040d5a9732c81ce1dd077`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:30.190357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fd51693a-4630-456d-9535-40beb91c07e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:30.204618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:30.281849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1157/2302] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:30.293046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad65db-2d70-4ea0-a212-5c7d89aa75bd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:30.353111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:30.375782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad65db-2d70-4ea0-a212-5c7d89aa75bd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:30.383370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6c90a2b95c498494a73b43e7138905', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:30.384230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:30.385021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:30.385502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:30.485877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:30.487725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:30.513115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1146/2303] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:30.747546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c3cc-80ea-426d-9a83-47127a80781c None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:30.747831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c3cc-80ea-426d-9a83-47127a80781c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:30.749938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c3cc-80ea-426d-9a83-47127a80781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:30.780404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4843c3cc-80ea-426d-9a83-47127a80781c None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:30.781712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4843c3cc-80ea-426d-9a83-47127a80781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:30.899243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1158/2304] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:21:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:30.926988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:30.928810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:31.022927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720c073-cfce-4438-b981-0fb418e5c6b3 service glance] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:31.027975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0ec11-b9c6-4ed0-a619-634e39988e65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:31.028652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0ec11-b9c6-4ed0-a619-634e39988e65 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:31.028890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0ec11-b9c6-4ed0-a619-634e39988e65 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:31.029155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0ec11-b9c6-4ed0-a619-634e39988e65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:31.039513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0ec11-b9c6-4ed0-a619-634e39988e65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:31.041166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0ec11-b9c6-4ed0-a619-634e39988e65 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:31.053006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1147/2305] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:21:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:21:31.189974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579deba6-c4ea-4cef-85dc-248e89396054 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:31.190537 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579deba6-c4ea-4cef-85dc-248e89396054 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:31.191000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579deba6-c4ea-4cef-85dc-248e89396054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:31.232438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-579deba6-c4ea-4cef-85dc-248e89396054 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:31.238356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-579deba6-c4ea-4cef-85dc-248e89396054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:31.257792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1159/2306] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:21:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:31.288157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0ec11-b9c6-4ed0-a619-634e39988e65 service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:31.331130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0ec11-b9c6-4ed0-a619-634e39988e65 service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:31.347036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50daac8a-c1f6-4f87-a714-cafdc4a2b6e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:31.347748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daac8a-c1f6-4f87-a714-cafdc4a2b6e1 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:31.348101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daac8a-c1f6-4f87-a714-cafdc4a2b6e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:31.348441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daac8a-c1f6-4f87-a714-cafdc4a2b6e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:31.364100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50daac8a-c1f6-4f87-a714-cafdc4a2b6e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:31.366728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50daac8a-c1f6-4f87-a714-cafdc4a2b6e1 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:31.398532 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1148/2307] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:21:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:21:41.215978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=50 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:41.387027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=68 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:42.359569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66820-f238-4604-b14d-a971288913e6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:42.360166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66820-f238-4604-b14d-a971288913e6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:42.360166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66820-f238-4604-b14d-a971288913e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:42.380997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-ecf66820-f238-4604-b14d-a971288913e6 None None] MFA Rules not processed for user `3e8d388698c6493c83f83059727a6188`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:42.382427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ecf66820-f238-4604-b14d-a971288913e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:42.395154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1160/2308] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:21:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:52.374867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:04.328662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4ffe-7a2f-47e9-9367-dbd4a2771d83 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:04.329187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4ffe-7a2f-47e9-9367-dbd4a2771d83 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:04.329187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4ffe-7a2f-47e9-9367-dbd4a2771d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:04.354484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1ffa4ffe-7a2f-47e9-9367-dbd4a2771d83 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:04.356126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1ffa4ffe-7a2f-47e9-9367-dbd4a2771d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:04.366726 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1149/2309] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:22:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:22:04.382584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf66820-f238-4604-b14d-a971288913e6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:04.396557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf66820-f238-4604-b14d-a971288913e6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:04.402546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2402-a1cb-4e5b-9159-8fdf9579346b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:04.403481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2402-a1cb-4e5b-9159-8fdf9579346b service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:04.403941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2402-a1cb-4e5b-9159-8fdf9579346b service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:04.404217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2402-a1cb-4e5b-9159-8fdf9579346b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:04.414356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac2402-a1cb-4e5b-9159-8fdf9579346b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:04.416056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac2402-a1cb-4e5b-9159-8fdf9579346b service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:04.450545 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1161/2310] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:22:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:22:14.353019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:14.422390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:18.897167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301d0c5-aba4-4b2a-8700-f7d1b148d13f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:18.897733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301d0c5-aba4-4b2a-8700-f7d1b148d13f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:18.897733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301d0c5-aba4-4b2a-8700-f7d1b148d13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:18.917325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2301d0c5-aba4-4b2a-8700-f7d1b148d13f None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:18.918525 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2301d0c5-aba4-4b2a-8700-f7d1b148d13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:18.925769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1150/2311] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:22:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:22:18.937689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2402-a1cb-4e5b-9159-8fdf9579346b service swift] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:18.948711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2402-a1cb-4e5b-9159-8fdf9579346b service swift] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:18.953651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b52f2-2f15-4d73-89cb-ab63e3e66d74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:18.954360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b52f2-2f15-4d73-89cb-ab63e3e66d74 service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:18.954643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b52f2-2f15-4d73-89cb-ab63e3e66d74 service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:18.955697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b52f2-2f15-4d73-89cb-ab63e3e66d74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:18.963944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b52f2-2f15-4d73-89cb-ab63e3e66d74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:18.965592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b52f2-2f15-4d73-89cb-ab63e3e66d74 service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:18.974083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1162/2312] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:22:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:22:19.514263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc094ce-c46c-4842-8d06-9347cfef9fd8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:19.514263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc094ce-c46c-4842-8d06-9347cfef9fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:19.514263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc094ce-c46c-4842-8d06-9347cfef9fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:19.536243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fcc094ce-c46c-4842-8d06-9347cfef9fd8 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:19.537492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fcc094ce-c46c-4842-8d06-9347cfef9fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:19.544722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1151/2313] 10.222.0.95 () {58 vars in 967 bytes} [Wed Aug 31 19:22:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:22:19.560240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b52f2-2f15-4d73-89cb-ab63e3e66d74 service swift] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:19.571143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b52f2-2f15-4d73-89cb-ab63e3e66d74 service swift] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:19.576590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae0c47-f19d-49cb-9ddb-9ae2fca24632 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:19.577271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0c47-f19d-49cb-9ddb-9ae2fca24632 service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:19.577623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0c47-f19d-49cb-9ddb-9ae2fca24632 service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:19.577947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0c47-f19d-49cb-9ddb-9ae2fca24632 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:19.587633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae0c47-f19d-49cb-9ddb-9ae2fca24632 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:19.589313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae0c47-f19d-49cb-9ddb-9ae2fca24632 service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:19.598803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1163/2314] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:22:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:22:19.883937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6485-bca0-44b8-b5a7-60a86e6f85bf None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:19.884697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6485-bca0-44b8-b5a7-60a86e6f85bf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:19.884909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6485-bca0-44b8-b5a7-60a86e6f85bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:19.927397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b62d6485-bca0-44b8-b5a7-60a86e6f85bf None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:19.929019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b62d6485-bca0-44b8-b5a7-60a86e6f85bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:19.950294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1152/2315] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:22:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:22:19.959065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae0c47-f19d-49cb-9ddb-9ae2fca24632 service swift] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:19.973320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae0c47-f19d-49cb-9ddb-9ae2fca24632 service swift] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:19.990035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5395f4-41c7-41a6-a20a-a4c8a899b483 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:19.990666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5395f4-41c7-41a6-a20a-a4c8a899b483 service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:19.990951 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5395f4-41c7-41a6-a20a-a4c8a899b483 service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:19.991225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5395f4-41c7-41a6-a20a-a4c8a899b483 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:20.006950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5395f4-41c7-41a6-a20a-a4c8a899b483 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:20.008948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5395f4-41c7-41a6-a20a-a4c8a899b483 service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:20.019492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1164/2316] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:22:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:22:20.378363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7ab9e-e2dc-4d7a-a697-87eed5034378 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:20.378647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7ab9e-e2dc-4d7a-a697-87eed5034378 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:20.378922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7ab9e-e2dc-4d7a-a697-87eed5034378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:20.397982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1df7ab9e-e2dc-4d7a-a697-87eed5034378 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:20.399104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1df7ab9e-e2dc-4d7a-a697-87eed5034378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:20.406285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1153/2317] 10.222.0.95 () {58 vars in 967 bytes} [Wed Aug 31 19:22:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:22:20.419460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5395f4-41c7-41a6-a20a-a4c8a899b483 service swift] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:20.433406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5395f4-41c7-41a6-a20a-a4c8a899b483 service swift] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:20.438950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaf069-ddac-48af-940a-8ee033705b17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:20.439763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf069-ddac-48af-940a-8ee033705b17 service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:20.440046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf069-ddac-48af-940a-8ee033705b17 service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:20.440342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf069-ddac-48af-940a-8ee033705b17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:20.448532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaf069-ddac-48af-940a-8ee033705b17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:20.450108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaf069-ddac-48af-940a-8ee033705b17 service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:20.458057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1165/2318] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:22:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:22:22.586598 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7ab9e-e2dc-4d7a-a697-87eed5034378 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:22.588071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1df7ab9e-e2dc-4d7a-a697-87eed5034378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:22.654569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7ab9e-e2dc-4d7a-a697-87eed5034378 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:22.659603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:22.660239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:22.660814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:22.661088 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] PATH_INFO: `/v3/users/c5efa16ebc44403b8382ecc784e1eddf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:22.662241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5efa16ebc44403b8382ecc784e1eddf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:22.664194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:22.712325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5efa16ebc44403b8382ecc784e1eddf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:22.725082 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5efa16ebc44403b8382ecc784e1eddf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:22.726589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5efa16ebc44403b8382ecc784e1eddf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:22.726972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5efa16ebc44403b8382ecc784e1eddf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:22.734641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5efa16ebc44403b8382ecc784e1eddf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:22.747058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1154/2319] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:22:22 2022] DELETE /identity/v3/users/c5efa16ebc44403b8382ecc784e1eddf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:22:22.752324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629666b-d1c5-4526-a012-a227a7933657 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:22.752324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629666b-d1c5-4526-a012-a227a7933657 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:22.753238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629666b-d1c5-4526-a012-a227a7933657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:22.772296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-5629666b-d1c5-4526-a012-a227a7933657 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:22.773549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5629666b-d1c5-4526-a012-a227a7933657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:22.832204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1166/2320] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:22:22.842026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:22.843442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:22.901106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf93975-1530-4b38-86c3-86501bb036d7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:22.904766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd72cf-eb50-4b73-a971-68fb815e7280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:22.905367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd72cf-eb50-4b73-a971-68fb815e7280 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:22.905659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd72cf-eb50-4b73-a971-68fb815e7280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:22.905922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd72cf-eb50-4b73-a971-68fb815e7280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:22.913173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:22.915515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd72cf-eb50-4b73-a971-68fb815e7280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:22.917998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd72cf-eb50-4b73-a971-68fb815e7280 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:22.928293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1155/2321] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:23.097856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629666b-d1c5-4526-a012-a227a7933657 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:23.099286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5629666b-d1c5-4526-a012-a227a7933657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:23.111653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629666b-d1c5-4526-a012-a227a7933657 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:23.117669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:23.118334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:23.118615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:23.118887 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] PATH_INFO: `/v3/projects/d9f01988234948f18e7182e829cbcf0b` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:23.120137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f01988234948f18e7182e829cbcf0b)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:23.122623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:23.178741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f01988234948f18e7182e829cbcf0b', 'request_id': 'req-a0a6585f-2703-424a-bbc1-038c1c91e877'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:23.184849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f01988234948f18e7182e829cbcf0b', 'request_id': 'req-a0a6585f-2703-424a-bbc1-038c1c91e877'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:23.191280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f01988234948f18e7182e829cbcf0b', 'request_id': 'req-a0a6585f-2703-424a-bbc1-038c1c91e877'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:23.203391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] The token cache is being invalidate because project d9f01988234948f18e7182e829cbcf0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:22:23.203733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:23.204406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:23.206041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1167/2322] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:22:23 2022] DELETE /identity/v3/projects/d9f01988234948f18e7182e829cbcf0b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:22:24.883166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43955b-055f-4f13-bbe1-2d13a7f1bf6a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:24.886991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43955b-055f-4f13-bbe1-2d13a7f1bf6a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:24.886991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43955b-055f-4f13-bbe1-2d13a7f1bf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:24.903608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ed43955b-055f-4f13-bbe1-2d13a7f1bf6a None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:24.904850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ed43955b-055f-4f13-bbe1-2d13a7f1bf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:24.961890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1156/2323] 10.222.0.95 () {58 vars in 967 bytes} [Wed Aug 31 19:22:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:22:24.985874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:24.988039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:25.025436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:25.064243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6585f-2703-424a-bbc1-038c1c91e877 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:25.069765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82af91-005d-4e21-a941-0e088c6db241 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:25.070375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82af91-005d-4e21-a941-0e088c6db241 service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:25.070656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82af91-005d-4e21-a941-0e088c6db241 service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:25.070928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82af91-005d-4e21-a941-0e088c6db241 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:25.080283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82af91-005d-4e21-a941-0e088c6db241 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:25.081958 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82af91-005d-4e21-a941-0e088c6db241 service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:25.091884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1168/2324] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:22:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:22:31.603016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43955b-055f-4f13-bbe1-2d13a7f1bf6a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:31.604552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ed43955b-055f-4f13-bbe1-2d13a7f1bf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:31.663952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43955b-055f-4f13-bbe1-2d13a7f1bf6a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:31.667932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:31.669250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:31.669250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:31.669492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:31.671455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:31.747068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:31.748667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:31.759948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1157/2325] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:32.934527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82af91-005d-4e21-a941-0e088c6db241 service swift] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:32.948708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82af91-005d-4e21-a941-0e088c6db241 service swift] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:32.952715 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:32.953320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:32.953601 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:32.953921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:32.955808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:32.969741 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:32.971242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:32.983038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1169/2326] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:41.755033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=81 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:42.977786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=24 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:49.031521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:49.033401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:49.047720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdfb3c-a1bd-4930-8ccc-0c3f7c36e69e service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:49.053911 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:49.054506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:49.054808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:49.055075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:49.056789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:49.118865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:49.120489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:49.130967 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1158/2327] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:49.613817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:49.616681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:49.630187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe3e41-6e09-4075-92de-159f896b10ea service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:49.636540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44356569-e288-48aa-ada5-676b019c439f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:49.637290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356569-e288-48aa-ada5-676b019c439f service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:49.638052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356569-e288-48aa-ada5-676b019c439f service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:49.638537 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356569-e288-48aa-ada5-676b019c439f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:49.640692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-44356569-e288-48aa-ada5-676b019c439f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:49.711307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44356569-e288-48aa-ada5-676b019c439f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:49.713044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44356569-e288-48aa-ada5-676b019c439f service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:49.723532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1170/2328] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:22:51.077833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:51.088510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beada7bf-4ae6-4762-ae4f-49d5fcd70b84 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:51.092110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d8480-077a-4e26-be09-4e28ce6b4019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:51.092749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8480-077a-4e26-be09-4e28ce6b4019 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:51.093033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8480-077a-4e26-be09-4e28ce6b4019 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:51.093307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d8480-077a-4e26-be09-4e28ce6b4019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:51.095085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-240d8480-077a-4e26-be09-4e28ce6b4019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:51.162174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d8480-077a-4e26-be09-4e28ce6b4019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:51.163549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d8480-077a-4e26-be09-4e28ce6b4019 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:51.174244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1159/2329] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:59.718422 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=28 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:01.169168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=58 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:06.867311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44356569-e288-48aa-ada5-676b019c439f service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:06.883421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44356569-e288-48aa-ada5-676b019c439f service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:06.894934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:06.895687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:06.896035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:06.896930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:06.899590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:06.937322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bcbbd-a68f-4241-8eaa-7bad1350a94d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:06.937880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bcbbd-a68f-4241-8eaa-7bad1350a94d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:06.937880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bcbbd-a68f-4241-8eaa-7bad1350a94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:06.962512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6f0bcbbd-a68f-4241-8eaa-7bad1350a94d None None] MFA Rules not processed for user `6e09978ed8b3490db5f92823768f1d15`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:06.963846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6f0bcbbd-a68f-4241-8eaa-7bad1350a94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:06.972359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1160/2330] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:23:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:23:06.981489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:06.983480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:06.995974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1171/2331] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:07.364821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bcbbd-a68f-4241-8eaa-7bad1350a94d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:07.382508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bcbbd-a68f-4241-8eaa-7bad1350a94d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:07.387732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3705e-7a86-45c3-878c-83e58e4eb859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:07.388662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3705e-7a86-45c3-878c-83e58e4eb859 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:07.389052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3705e-7a86-45c3-878c-83e58e4eb859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:07.389339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3705e-7a86-45c3-878c-83e58e4eb859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:07.392960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-88e3705e-7a86-45c3-878c-83e58e4eb859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:07.408973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3705e-7a86-45c3-878c-83e58e4eb859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:07.411379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3705e-7a86-45c3-878c-83e58e4eb859 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:07.425576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1161/2332] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:23:08.100086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:08.101657 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:08.180486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecf4a6-95a3-4a03-9ee7-6c7b8c87d8ab service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:08.186138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:08.186771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:08.187051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:08.187318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:08.190228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:08.202076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:08.205236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:08.216056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1172/2333] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:10.310003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c720f-9f9d-4a52-ba29-79a760bef89b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:10.310490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c720f-9f9d-4a52-ba29-79a760bef89b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:10.310581 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c720f-9f9d-4a52-ba29-79a760bef89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:10.332821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dd8c720f-9f9d-4a52-ba29-79a760bef89b None None] MFA Rules not processed for user `1e04230bb5b34b27aad5c2a46230a5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:10.334019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dd8c720f-9f9d-4a52-ba29-79a760bef89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:10.341826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:10.344295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:10.357667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276748-26be-4714-aef2-e9a818e5ef39 service placement] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:10.362045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:10.362669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:10.362948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:10.363217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:10.365092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:10.375070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:10.377309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:10.386050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1173/2334] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:23:10.397892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1162/2335] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:23:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:10.724498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:10.733486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4939f-70b9-4c0e-84cf-cc37c7161d9f service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:10.737918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9739f8-9a52-41df-add3-6fb168943a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:10.738526 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9739f8-9a52-41df-add3-6fb168943a79 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:10.738803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9739f8-9a52-41df-add3-6fb168943a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:10.739128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9739f8-9a52-41df-add3-6fb168943a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:10.741594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-da9739f8-9a52-41df-add3-6fb168943a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:10.751811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9739f8-9a52-41df-add3-6fb168943a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:10.753298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9739f8-9a52-41df-add3-6fb168943a79 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:10.761681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1174/2336] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:14.167796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c720f-9f9d-4a52-ba29-79a760bef89b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:14.183056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c720f-9f9d-4a52-ba29-79a760bef89b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:14.189691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64812b-9d42-4bb7-8eaf-784c6c1e6dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:14.191053 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64812b-9d42-4bb7-8eaf-784c6c1e6dbc service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:14.192407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64812b-9d42-4bb7-8eaf-784c6c1e6dbc service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:14.192712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64812b-9d42-4bb7-8eaf-784c6c1e6dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:14.195025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0f64812b-9d42-4bb7-8eaf-784c6c1e6dbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:14.212547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64812b-9d42-4bb7-8eaf-784c6c1e6dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:14.214195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64812b-9d42-4bb7-8eaf-784c6c1e6dbc service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:14.228283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1163/2337] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:15.893121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea2870-cb4a-49de-a25f-307b6d77837e None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:15.893695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea2870-cb4a-49de-a25f-307b6d77837e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:15.893695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea2870-cb4a-49de-a25f-307b6d77837e None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:15.895135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1175/2338] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:23:15.912633 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af259147-5833-4e25-8527-1a7d2287f43b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:15.913653 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af259147-5833-4e25-8527-1a7d2287f43b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:15.914156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af259147-5833-4e25-8527-1a7d2287f43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:15.943566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-af259147-5833-4e25-8527-1a7d2287f43b None None] MFA Rules not processed for user `5f1d5c29df30465ead9834b63c3e93bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:15.945430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-af259147-5833-4e25-8527-1a7d2287f43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:15.963226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1164/2339] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:23:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:15.999099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea2870-cb4a-49de-a25f-307b6d77837e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:16.012307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea2870-cb4a-49de-a25f-307b6d77837e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:16.017421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d946e-1e6f-4165-8b10-07b674cb165a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:16.018260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d946e-1e6f-4165-8b10-07b674cb165a service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:16.018613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d946e-1e6f-4165-8b10-07b674cb165a service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:16.018948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d946e-1e6f-4165-8b10-07b674cb165a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:16.029036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d946e-1e6f-4165-8b10-07b674cb165a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:16.031015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d946e-1e6f-4165-8b10-07b674cb165a service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:16.043953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1176/2340] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:23:25.933843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=43 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:26.038491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=78 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:28.101023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af259147-5833-4e25-8527-1a7d2287f43b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:28.114534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af259147-5833-4e25-8527-1a7d2287f43b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:28.120612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea93a69-578d-4271-baed-fd739559694b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:28.121472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93a69-578d-4271-baed-fd739559694b service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:28.121969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93a69-578d-4271-baed-fd739559694b service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:28.122451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea93a69-578d-4271-baed-fd739559694b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:28.124432 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3ea93a69-578d-4271-baed-fd739559694b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:28.191919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea93a69-578d-4271-baed-fd739559694b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:28.194665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea93a69-578d-4271-baed-fd739559694b service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:28.204325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1165/2341] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:32.388360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d946e-1e6f-4165-8b10-07b674cb165a service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:32.404615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d946e-1e6f-4165-8b10-07b674cb165a service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:32.416609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01132ef4-4ea8-4014-9b77-aa4b4b7c039f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:32.417471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01132ef4-4ea8-4014-9b77-aa4b4b7c039f service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:32.417760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01132ef4-4ea8-4014-9b77-aa4b4b7c039f service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:32.418031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01132ef4-4ea8-4014-9b77-aa4b4b7c039f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:32.419816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-01132ef4-4ea8-4014-9b77-aa4b4b7c039f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:32.434096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01132ef4-4ea8-4014-9b77-aa4b4b7c039f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:32.435727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01132ef4-4ea8-4014-9b77-aa4b4b7c039f service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:32.448218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1177/2342] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:38.203069 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=28 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:42.444236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:50.052971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f875b-5d1c-4d42-a16f-33f20cf71ebe None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:50.053502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f875b-5d1c-4d42-a16f-33f20cf71ebe None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:50.053608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f875b-5d1c-4d42-a16f-33f20cf71ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:50.074278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d45f875b-5d1c-4d42-a16f-33f20cf71ebe None None] MFA Rules not processed for user `25c3f0a2e1fd4067b1804ef42faa46d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:50.075575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d45f875b-5d1c-4d42-a16f-33f20cf71ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:50.083899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1166/2343] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:23:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:50.463872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf76a3a-5073-43da-97bf-439c3dbb6cef None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:50.464900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf76a3a-5073-43da-97bf-439c3dbb6cef None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:50.465401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf76a3a-5073-43da-97bf-439c3dbb6cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:50.488011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-1cf76a3a-5073-43da-97bf-439c3dbb6cef None None] MFA Rules not processed for user `fd0f7f51482040d7a89df3f7502c2638`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:50.489260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1cf76a3a-5073-43da-97bf-439c3dbb6cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:50.498220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1178/2344] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:23:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:59.786524 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f875b-5d1c-4d42-a16f-33f20cf71ebe None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:59.801841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f875b-5d1c-4d42-a16f-33f20cf71ebe None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:59.810219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:59.810219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:59.810696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:59.811629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:59.813672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:59.840748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:59.846760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:59.875853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1167/2345] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:00.481804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:09.860108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=9 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:22.055107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568068a2-d37b-49f4-a1af-1c8635572c28 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:22.055638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568068a2-d37b-49f4-a1af-1c8635572c28 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:22.055744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568068a2-d37b-49f4-a1af-1c8635572c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:22.076628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-568068a2-d37b-49f4-a1af-1c8635572c28 None None] MFA Rules not processed for user `a288c513afd94adaa9f53f9f65dc9dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:22.078615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-568068a2-d37b-49f4-a1af-1c8635572c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:22.085848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1179/2346] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:25.109661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:25.111958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:25.181680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee90be-3a8a-4e69-84cc-babd1bfd1263 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:25.186522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:25.187154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:25.187436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:25.187704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] PATH_INFO: `/v3/users/1e04230bb5b34b27aad5c2a46230a5cc` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:25.188854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e04230bb5b34b27aad5c2a46230a5cc)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:25.190751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:25.245900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e04230bb5b34b27aad5c2a46230a5cc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.260964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e04230bb5b34b27aad5c2a46230a5cc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.261708 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e04230bb5b34b27aad5c2a46230a5cc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.262493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e04230bb5b34b27aad5c2a46230a5cc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.270588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d794de01-383b-4f40-8ec3-eb67c7f4aff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e04230bb5b34b27aad5c2a46230a5cc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.284828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1168/2347] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:25 2022] DELETE /identity/v3/users/1e04230bb5b34b27aad5c2a46230a5cc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:25.747306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568068a2-d37b-49f4-a1af-1c8635572c28 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:25.763021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568068a2-d37b-49f4-a1af-1c8635572c28 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:25.830634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:25.831666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:25.832018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:25.832361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] PATH_INFO: `/v3/projects/3b530b6cef6943ee93e561b05d1f846d` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:25.833643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b530b6cef6943ee93e561b05d1f846d)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:25.835717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:25.886751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b530b6cef6943ee93e561b05d1f846d', 'request_id': 'req-9d3c6f90-34a3-4e4e-839b-3b137dad3098'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.893578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b530b6cef6943ee93e561b05d1f846d', 'request_id': 'req-9d3c6f90-34a3-4e4e-839b-3b137dad3098'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.899367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b530b6cef6943ee93e561b05d1f846d', 'request_id': 'req-9d3c6f90-34a3-4e4e-839b-3b137dad3098'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.915498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] The token cache is being invalidate because project 3b530b6cef6943ee93e561b05d1f846d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:25.915960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.916835 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:25.919132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1180/2348] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:25 2022] DELETE /identity/v3/projects/3b530b6cef6943ee93e561b05d1f846d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:35.281192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:35.910616 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:56.559760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768ed17-2aea-4cac-9d5f-8b08b7f21d7f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:56.560313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768ed17-2aea-4cac-9d5f-8b08b7f21d7f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:56.560419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768ed17-2aea-4cac-9d5f-8b08b7f21d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:56.580503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6768ed17-2aea-4cac-9d5f-8b08b7f21d7f None None] MFA Rules not processed for user `6e09978ed8b3490db5f92823768f1d15`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:56.581760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6768ed17-2aea-4cac-9d5f-8b08b7f21d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:56.639795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1169/2349] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:24:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:06.632769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=25 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:24.417996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:24.420058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:24.534877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6f90-34a3-4e4e-839b-3b137dad3098 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:24.539052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:24.539805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:24.540189 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:24.540557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:24.542973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:24.669922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:24.671393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:24.680354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1181/2350] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:32.487777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2f7a1-11c0-4540-a25d-2052ee32c5e3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:32.488296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2f7a1-11c0-4540-a25d-2052ee32c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:32.488389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2f7a1-11c0-4540-a25d-2052ee32c5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:32.521323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-62f2f7a1-11c0-4540-a25d-2052ee32c5e3 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:32.522526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-62f2f7a1-11c0-4540-a25d-2052ee32c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:32.586567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1170/2351] 10.222.0.95 () {58 vars in 967 bytes} [Wed Aug 31 19:25:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:32.637938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:32.639399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:32.714696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b59c2-a673-4887-beda-a9c67d5a91f7 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:32.720099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2133b5-4b0a-4c2e-a6e0-4cb00bbc4b75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:32.720824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2133b5-4b0a-4c2e-a6e0-4cb00bbc4b75 service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:32.721108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2133b5-4b0a-4c2e-a6e0-4cb00bbc4b75 service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:32.721376 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2133b5-4b0a-4c2e-a6e0-4cb00bbc4b75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:32.727507 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:32.730126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2133b5-4b0a-4c2e-a6e0-4cb00bbc4b75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:32.732293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2133b5-4b0a-4c2e-a6e0-4cb00bbc4b75 service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:32.741256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1182/2352] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:25:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:25:33.997885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2f7a1-11c0-4540-a25d-2052ee32c5e3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:33.999602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-62f2f7a1-11c0-4540-a25d-2052ee32c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:34.013448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2f7a1-11c0-4540-a25d-2052ee32c5e3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:34.019196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:34.020314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:34.020795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:34.021517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:34.023458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:34.137871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:34.140473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:34.170987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1171/2353] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:36.068629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2133b5-4b0a-4c2e-a6e0-4cb00bbc4b75 service swift] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.079875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2133b5-4b0a-4c2e-a6e0-4cb00bbc4b75 service swift] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:36.086692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:36.087321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:36.087607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:36.087879 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:36.097420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:36.131301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:36.134074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:36.152070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1183/2354] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:36.959977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:36.961686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:37.066733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:37.104398 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fc2d2-e640-4379-aab4-c866d8fcad9a service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:37.112817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:37.114995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:37.115477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:37.116049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:37.120026 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:37.138084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:37.140170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:37.152693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1172/2355] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:40.421284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:40.434764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16e89f-9283-42c4-8b16-b7ffef62bafd service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:40.444066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:40.445128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:40.445543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:40.445916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:40.448449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:40.464993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:40.467173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:40.488687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1184/2356] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:40.499275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:40.502057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:40.530670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852e7ca-0e66-4861-b295-788a3b9cd039 service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:40.538485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:40.540332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:40.540716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:40.541076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] PATH_INFO: `/v3/users/fd0f7f51482040d7a89df3f7502c2638` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:40.542825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0f7f51482040d7a89df3f7502c2638)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:40.545835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:40.636849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0f7f51482040d7a89df3f7502c2638'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.651299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0f7f51482040d7a89df3f7502c2638'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.651958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0f7f51482040d7a89df3f7502c2638'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.652602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0f7f51482040d7a89df3f7502c2638'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.663810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7006745-77d5-4832-9113-cfca9053c45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0f7f51482040d7a89df3f7502c2638'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:40.678632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1173/2357] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:40 2022] DELETE /identity/v3/users/fd0f7f51482040d7a89df3f7502c2638 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:40.921579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:40.939769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53f0ca-7fcd-4e5e-b703-1ec905960f2c service placement] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.028551 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.029621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.030297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.030762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] PATH_INFO: `/v3/projects/a1342c2dbb1c41dca6061505220d9de7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.033199 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1342c2dbb1c41dca6061505220d9de7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.035432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:41.123992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1342c2dbb1c41dca6061505220d9de7', 'request_id': 'req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.134254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1342c2dbb1c41dca6061505220d9de7', 'request_id': 'req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.155139 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1342c2dbb1c41dca6061505220d9de7', 'request_id': 'req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.172411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] The token cache is being invalidate because project a1342c2dbb1c41dca6061505220d9de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:41.173669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.174623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-97c1aa34-41cf-4709-ae72-373e6e5b3b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.176733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1185/2358] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:40 2022] DELETE /identity/v3/projects/a1342c2dbb1c41dca6061505220d9de7 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:41.181896 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469f08d-32f6-453a-8396-2d7aba51d55b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.182185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469f08d-32f6-453a-8396-2d7aba51d55b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.182631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469f08d-32f6-453a-8396-2d7aba51d55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.288046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5469f08d-32f6-453a-8396-2d7aba51d55b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:41.289528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5469f08d-32f6-453a-8396-2d7aba51d55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:41.387380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1174/2359] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:41.396745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0dbda-3194-4264-b86b-50eb8f5610d2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.397038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0dbda-3194-4264-b86b-50eb8f5610d2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.397315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0dbda-3194-4264-b86b-50eb8f5610d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.434986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-9bb0dbda-3194-4264-b86b-50eb8f5610d2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:41.436701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9bb0dbda-3194-4264-b86b-50eb8f5610d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:41.450090 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1186/2360] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:41.451817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469f08d-32f6-453a-8396-2d7aba51d55b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:41.466268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469f08d-32f6-453a-8396-2d7aba51d55b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.473788 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.474568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.474955 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.475323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.476272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:41.476802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.480040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:41.483297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:41.505979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1175/2361] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:25:41.507548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0dbda-3194-4264-b86b-50eb8f5610d2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:41.526310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0dbda-3194-4264-b86b-50eb8f5610d2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.533030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.533229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.533928 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.534445 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.535179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.537366 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:41.592200 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1187/2362] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:41 2022] POST /identity/v3/projects => generated 405 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:41.596368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:41.607615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0fda-b955-4b00-a4a4-c311ab3fe477 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.707522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61694b-4d25-4a37-9a7b-b115357e12da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.708649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61694b-4d25-4a37-9a7b-b115357e12da admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.708649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61694b-4d25-4a37-9a7b-b115357e12da admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.708883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61694b-4d25-4a37-9a7b-b115357e12da admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.709802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61694b-4d25-4a37-9a7b-b115357e12da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.712270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61694b-4d25-4a37-9a7b-b115357e12da admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:41.747497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:41.750290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:41.778680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1176/2363] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:41 2022] POST /identity/v3/users => generated 303 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:41.783248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61694b-4d25-4a37-9a7b-b115357e12da admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:41.787597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77916e-8d7c-4d1c-bb22-da85b9613836 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.793233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.794745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61694b-4d25-4a37-9a7b-b115357e12da admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.798450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.798837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.799123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] PATH_INFO: `/v3/users/3e8d388698c6493c83f83059727a6188` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.801789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8d388698c6493c83f83059727a6188)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.806840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b9efd-ea6c-4f01-946a-07dd2de13b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.807478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b9efd-ea6c-4f01-946a-07dd2de13b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.807763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b9efd-ea6c-4f01-946a-07dd2de13b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.808100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b9efd-ea6c-4f01-946a-07dd2de13b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.808849 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b9efd-ea6c-4f01-946a-07dd2de13b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.810652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b9efd-ea6c-4f01-946a-07dd2de13b36 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:41.813027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:41.824659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1177/2364] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:41.829851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b9efd-ea6c-4f01-946a-07dd2de13b36 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:41.845317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b9efd-ea6c-4f01-946a-07dd2de13b36 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.852327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:41.857373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424603c4-2c08-4954-8a7f-e8ed10abd456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.858127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424603c4-2c08-4954-8a7f-e8ed10abd456 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.858499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424603c4-2c08-4954-8a7f-e8ed10abd456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.858891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424603c4-2c08-4954-8a7f-e8ed10abd456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.860297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424603c4-2c08-4954-8a7f-e8ed10abd456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.868018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424603c4-2c08-4954-8a7f-e8ed10abd456 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:41.886170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1178/2365] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:41.887239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8d388698c6493c83f83059727a6188'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.890130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424603c4-2c08-4954-8a7f-e8ed10abd456 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:41.896520 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8d388698c6493c83f83059727a6188'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.897935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8d388698c6493c83f83059727a6188'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.901653 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424603c4-2c08-4954-8a7f-e8ed10abd456 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:41.912985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:41.913693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.913973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.914437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] PATH_INFO: `/v3/projects/803aaa37e02f4e259e43eb3bc19ba9b5/users/011c9c7cbb2a49daa8d61ae247472b4b/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:41.916607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8d388698c6493c83f83059727a6188'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.916607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5710433e-c985-47f4-a1e2-8bb06e857303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8d388698c6493c83f83059727a6188'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:41.936611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1188/2366] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:41 2022] DELETE /identity/v3/users/3e8d388698c6493c83f83059727a6188 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:41.937580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803aaa37e02f4e259e43eb3bc19ba9b5, user_id=011c9c7cbb2a49daa8d61ae247472b4b, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:41.940770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:41.968331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1179/2367] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:41 2022] PUT /identity/v3/projects/803aaa37e02f4e259e43eb3bc19ba9b5/users/011c9c7cbb2a49daa8d61ae247472b4b/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:41.974006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876103d-be84-4a87-9dc8-7b3b1bab1984 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:41.975477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876103d-be84-4a87-9dc8-7b3b1bab1984 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:41.975760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876103d-be84-4a87-9dc8-7b3b1bab1984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.002434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-1876103d-be84-4a87-9dc8-7b3b1bab1984 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:42.003687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1876103d-be84-4a87-9dc8-7b3b1bab1984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:42.080502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1189/2368] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:42.090644 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:42.092637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:42.176954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604929f-1b60-4775-a3a4-1bcfc6ae4dc5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:42.180842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915114d-5b30-4b08-b41d-a6a76250e47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:42.181448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915114d-5b30-4b08-b41d-a6a76250e47b service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:42.181728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915114d-5b30-4b08-b41d-a6a76250e47b service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:42.181998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915114d-5b30-4b08-b41d-a6a76250e47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.193647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915114d-5b30-4b08-b41d-a6a76250e47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:42.195284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915114d-5b30-4b08-b41d-a6a76250e47b service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:42.207248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1180/2369] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:42.301932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876103d-be84-4a87-9dc8-7b3b1bab1984 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:42.303525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1876103d-be84-4a87-9dc8-7b3b1bab1984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:42.319802 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876103d-be84-4a87-9dc8-7b3b1bab1984 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:42.323426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:42.324215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:42.324546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:42.325006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] PATH_INFO: `/v3/projects/a16d7c3be2644b64b9d43e77d9e45804` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.326392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16d7c3be2644b64b9d43e77d9e45804)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:42.328678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:42.392739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16d7c3be2644b64b9d43e77d9e45804', 'request_id': 'req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.401003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16d7c3be2644b64b9d43e77d9e45804', 'request_id': 'req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.411238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16d7c3be2644b64b9d43e77d9e45804', 'request_id': 'req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.433678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] The token cache is being invalidate because project a16d7c3be2644b64b9d43e77d9e45804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:42.434273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.435234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-6e1a9f02-902b-43f3-b5ef-1ebd667e975e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.437423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1190/2370] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:42 2022] DELETE /identity/v3/projects/a16d7c3be2644b64b9d43e77d9e45804 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:42.442256 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915114d-5b30-4b08-b41d-a6a76250e47b service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:42.455360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915114d-5b30-4b08-b41d-a6a76250e47b service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:42.509701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:42.537669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:42.538316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:42.538617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:42.538866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] PATH_INFO: `/v3/users/1b94bf5be4e040d5a9732c81ce1dd077` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:42.540064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b94bf5be4e040d5a9732c81ce1dd077)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:42.542230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:42.637057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b94bf5be4e040d5a9732c81ce1dd077'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.659537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b94bf5be4e040d5a9732c81ce1dd077'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.660786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b94bf5be4e040d5a9732c81ce1dd077'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.661857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b94bf5be4e040d5a9732c81ce1dd077'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.676539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b94bf5be4e040d5a9732c81ce1dd077'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:42.712871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1181/2371] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:42 2022] DELETE /identity/v3/users/1b94bf5be4e040d5a9732c81ce1dd077 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:43.104660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5eaed-d3fe-4a1d-9b93-d7bb2d2cfb2f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.105407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5eaed-d3fe-4a1d-9b93-d7bb2d2cfb2f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.105665 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5eaed-d3fe-4a1d-9b93-d7bb2d2cfb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.117482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.119030 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:43.182678 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-cdc5eaed-d3fe-4a1d-9b93-d7bb2d2cfb2f None None] MFA Rules not processed for user `011c9c7cbb2a49daa8d61ae247472b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:43.184215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cdc5eaed-d3fe-4a1d-9b93-d7bb2d2cfb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:43.198371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:43.232670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f955a-21f2-4bf9-8b33-e5780acca098 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.241351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.242121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.242505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.242876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] PATH_INFO: `/v3/projects/1e249e3d4fea41b6a97aa8f2e060bbd1` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.244360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e249e3d4fea41b6a97aa8f2e060bbd1)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.246550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.273089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1191/2372] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:43 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:43.274688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5eaed-d3fe-4a1d-9b93-d7bb2d2cfb2f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.276369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cdc5eaed-d3fe-4a1d-9b93-d7bb2d2cfb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:43.295871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5eaed-d3fe-4a1d-9b93-d7bb2d2cfb2f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.307899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e249e3d4fea41b6a97aa8f2e060bbd1', 'request_id': 'req-cafe7e9d-c56d-4689-8fbb-214a33d43286'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:43.321997 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e249e3d4fea41b6a97aa8f2e060bbd1', 'request_id': 'req-cafe7e9d-c56d-4689-8fbb-214a33d43286'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:43.334303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e249e3d4fea41b6a97aa8f2e060bbd1', 'request_id': 'req-cafe7e9d-c56d-4689-8fbb-214a33d43286'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:43.335924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:43.349080 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] The token cache is being invalidate because project 1e249e3d4fea41b6a97aa8f2e060bbd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:43.350035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:43.351022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:43.355276 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1182/2373] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:43 2022] DELETE /identity/v3/projects/1e249e3d4fea41b6a97aa8f2e060bbd1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:43.382914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abaaf6-ef62-483c-ba4a-0ea14952bf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.383557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abaaf6-ef62-483c-ba4a-0ea14952bf87 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.383840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abaaf6-ef62-483c-ba4a-0ea14952bf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.384121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abaaf6-ef62-483c-ba4a-0ea14952bf87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.384810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abaaf6-ef62-483c-ba4a-0ea14952bf87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.387033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abaaf6-ef62-483c-ba4a-0ea14952bf87 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.426119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1192/2374] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:43 2022] POST /identity/v3/projects => generated 405 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:43.428331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.430529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:43.445501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe7e9d-c56d-4689-8fbb-214a33d43286 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.517610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284af67-e070-49e8-a65e-8eb72d103305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.518296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284af67-e070-49e8-a65e-8eb72d103305 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.518538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284af67-e070-49e8-a65e-8eb72d103305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.518766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284af67-e070-49e8-a65e-8eb72d103305 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.519415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284af67-e070-49e8-a65e-8eb72d103305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.521558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284af67-e070-49e8-a65e-8eb72d103305 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.549649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1183/2375] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:43 2022] POST /identity/v3/users => generated 309 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:43.554343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abaaf6-ef62-483c-ba4a-0ea14952bf87 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.566767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abaaf6-ef62-483c-ba4a-0ea14952bf87 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.573163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd281f1-9d13-4451-83ce-f18361332050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.573858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd281f1-9d13-4451-83ce-f18361332050 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.574289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd281f1-9d13-4451-83ce-f18361332050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.574749 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd281f1-9d13-4451-83ce-f18361332050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.575787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd281f1-9d13-4451-83ce-f18361332050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.578924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd281f1-9d13-4451-83ce-f18361332050 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.597752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1193/2376] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:43.601923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284af67-e070-49e8-a65e-8eb72d103305 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.615155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284af67-e070-49e8-a65e-8eb72d103305 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.619968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d35edf-11af-4753-95d5-ce857ff2cdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.620677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d35edf-11af-4753-95d5-ce857ff2cdef admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.621121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d35edf-11af-4753-95d5-ce857ff2cdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.621409 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d35edf-11af-4753-95d5-ce857ff2cdef admin admin] PATH_INFO: `/v3/projects/ca2aad0f7b1749478a3824ef4c30f370/users/34bd18f8380d460889a4ebd665f84aa2/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.643381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d35edf-11af-4753-95d5-ce857ff2cdef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2aad0f7b1749478a3824ef4c30f370, user_id=34bd18f8380d460889a4ebd665f84aa2, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.646650 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d35edf-11af-4753-95d5-ce857ff2cdef admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.670773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1184/2377] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:43 2022] PUT /identity/v3/projects/ca2aad0f7b1749478a3824ef4c30f370/users/34bd18f8380d460889a4ebd665f84aa2/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:43.675251 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd281f1-9d13-4451-83ce-f18361332050 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.694788 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd281f1-9d13-4451-83ce-f18361332050 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.764957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dbd6e-7dbb-4c71-8c59-bd2baf1ef515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.765657 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dbd6e-7dbb-4c71-8c59-bd2baf1ef515 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.766543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dbd6e-7dbb-4c71-8c59-bd2baf1ef515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.767290 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dbd6e-7dbb-4c71-8c59-bd2baf1ef515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.767599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2dbd6e-7dbb-4c71-8c59-bd2baf1ef515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.769373 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2dbd6e-7dbb-4c71-8c59-bd2baf1ef515 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.780623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1194/2378] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:43.786973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d35edf-11af-4753-95d5-ce857ff2cdef admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:43.799060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d35edf-11af-4753-95d5-ce857ff2cdef admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:43.804015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:43.804952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:43.805682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:43.805682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] PATH_INFO: `/v3/projects/ca2aad0f7b1749478a3824ef4c30f370/users/34bd18f8380d460889a4ebd665f84aa2/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:43.808019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2aad0f7b1749478a3824ef4c30f370, user_id=34bd18f8380d460889a4ebd665f84aa2, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:43.810837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:43.823714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1185/2379] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:43 2022] PUT /identity/v3/projects/ca2aad0f7b1749478a3824ef4c30f370/users/34bd18f8380d460889a4ebd665f84aa2/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:44.658737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf432e-256e-4bf9-baba-39aa1bc83f06 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:44.659345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf432e-256e-4bf9-baba-39aa1bc83f06 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:44.659433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf432e-256e-4bf9-baba-39aa1bc83f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:44.697640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c4bf432e-256e-4bf9-baba-39aa1bc83f06 None None] MFA Rules not processed for user `34bd18f8380d460889a4ebd665f84aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:44.699781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c4bf432e-256e-4bf9-baba-39aa1bc83f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:44.717431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:44.807348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1195/2380] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:44 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:46.720373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:46.723129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:46.810595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf432e-256e-4bf9-baba-39aa1bc83f06 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:46.836981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdd797-13ef-42df-b6b8-e1386326ed0d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:46.842776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:46.843705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:46.845053 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:46.845791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] PATH_INFO: `/v3/users/854d5db6c7f64422a946d5284570e1be` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:46.847395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=854d5db6c7f64422a946d5284570e1be)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:46.849715 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:46.892039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf432e-256e-4bf9-baba-39aa1bc83f06 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:46.898268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:46.898916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:46.899194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:46.899456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] PATH_INFO: `/v3/users/011c9c7cbb2a49daa8d61ae247472b4b` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:46.900631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=011c9c7cbb2a49daa8d61ae247472b4b)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:46.903399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:46.910046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:46.913477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854d5db6c7f64422a946d5284570e1be'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.930872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854d5db6c7f64422a946d5284570e1be'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.931680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854d5db6c7f64422a946d5284570e1be'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.932748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854d5db6c7f64422a946d5284570e1be'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.942657 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854d5db6c7f64422a946d5284570e1be'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.960611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1186/2381] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:46 2022] DELETE /identity/v3/users/854d5db6c7f64422a946d5284570e1be => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:46.971624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '011c9c7cbb2a49daa8d61ae247472b4b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.984970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '011c9c7cbb2a49daa8d61ae247472b4b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:46.985919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '011c9c7cbb2a49daa8d61ae247472b4b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.004864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '011c9c7cbb2a49daa8d61ae247472b4b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.005742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '011c9c7cbb2a49daa8d61ae247472b4b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.026166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1196/2382] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:46 2022] DELETE /identity/v3/users/011c9c7cbb2a49daa8d61ae247472b4b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:47.169697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.171217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.224618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.227383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.261296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e709879-3a71-4e50-8cb6-139f3bc8e477 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.282692 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:47.317099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb5184-c6e1-4b6c-b959-fa6ec8ab3d54 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.321690 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.322319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.322602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.322875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] PATH_INFO: `/v3/projects/803aaa37e02f4e259e43eb3bc19ba9b5` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.323980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=803aaa37e02f4e259e43eb3bc19ba9b5)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.324254 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.324920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.325203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.325477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] PATH_INFO: `/v3/projects/85c019992192497d9832594a889da9bf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.325963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.326672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c019992192497d9832594a889da9bf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.329236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.376449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803aaa37e02f4e259e43eb3bc19ba9b5', 'request_id': 'req-cdd3116f-16db-4c0d-af17-41278cb95129'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.378302 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c019992192497d9832594a889da9bf', 'request_id': 'req-bcd99006-c097-4f42-a39d-974b52e77ecf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.383522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803aaa37e02f4e259e43eb3bc19ba9b5', 'request_id': 'req-cdd3116f-16db-4c0d-af17-41278cb95129'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.386520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c019992192497d9832594a889da9bf', 'request_id': 'req-bcd99006-c097-4f42-a39d-974b52e77ecf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.391334 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803aaa37e02f4e259e43eb3bc19ba9b5', 'request_id': 'req-cdd3116f-16db-4c0d-af17-41278cb95129'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.393922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c019992192497d9832594a889da9bf', 'request_id': 'req-bcd99006-c097-4f42-a39d-974b52e77ecf'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.401333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:47.405049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] The token cache is being invalidate because project 803aaa37e02f4e259e43eb3bc19ba9b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:47.405412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.406163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.407954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] The token cache is being invalidate because project 85c019992192497d9832594a889da9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:47.408096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.408878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.409000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1197/2383] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:47 2022] DELETE /identity/v3/projects/803aaa37e02f4e259e43eb3bc19ba9b5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:47.411359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1187/2384] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:47 2022] DELETE /identity/v3/projects/85c019992192497d9832594a889da9bf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:47.413255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.414789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.414925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.416317 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.482447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3116f-16db-4c0d-af17-41278cb95129 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.483615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd99006-c097-4f42-a39d-974b52e77ecf admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.487132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.487779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.488065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.488363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] PATH_INFO: `/v3/users/34bd18f8380d460889a4ebd665f84aa2` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.489415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.489549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34bd18f8380d460889a4ebd665f84aa2)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.490031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.490334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.490603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] PATH_INFO: `/v3/users/080a65c2609a4d2bad406378af30bfa8` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.491441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.491727 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=080a65c2609a4d2bad406378af30bfa8)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.493672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.541442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34bd18f8380d460889a4ebd665f84aa2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.546792 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080a65c2609a4d2bad406378af30bfa8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.549414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34bd18f8380d460889a4ebd665f84aa2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.550179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34bd18f8380d460889a4ebd665f84aa2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.555482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:47.560922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080a65c2609a4d2bad406378af30bfa8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.561867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080a65c2609a4d2bad406378af30bfa8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.562520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080a65c2609a4d2bad406378af30bfa8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.565388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34bd18f8380d460889a4ebd665f84aa2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.566320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34bd18f8380d460889a4ebd665f84aa2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.571583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080a65c2609a4d2bad406378af30bfa8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.583996 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1198/2385] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:47 2022] DELETE /identity/v3/users/34bd18f8380d460889a4ebd665f84aa2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:47.585981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1188/2386] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:47 2022] DELETE /identity/v3/users/080a65c2609a4d2bad406378af30bfa8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:47.808106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.809570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.814924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:47.817327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:47.902029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10cf5c-3fee-4ec2-99f7-33d8276343e3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.907525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1107c01-0987-46b1-b73a-f7e6d1de0a37 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:47.910117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.911204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.911807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.912288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] PATH_INFO: `/v3/projects/ca2aad0f7b1749478a3824ef4c30f370` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.912646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:47.913283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:47.913566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:47.913837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] PATH_INFO: `/v3/projects/cea2433debb24b698dd2836acd25996e` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:47.914637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2aad0f7b1749478a3824ef4c30f370)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.914971 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea2433debb24b698dd2836acd25996e)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:47.916999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.918431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:47.947928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:47.985842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea2433debb24b698dd2836acd25996e', 'request_id': 'req-c60214ba-bc07-4f39-a74f-48209adc24a0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.993693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2aad0f7b1749478a3824ef4c30f370', 'request_id': 'req-5991592e-9389-476d-89a7-5720d017432a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:47.994846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea2433debb24b698dd2836acd25996e', 'request_id': 'req-c60214ba-bc07-4f39-a74f-48209adc24a0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:48.002105 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea2433debb24b698dd2836acd25996e', 'request_id': 'req-c60214ba-bc07-4f39-a74f-48209adc24a0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:48.012898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2aad0f7b1749478a3824ef4c30f370', 'request_id': 'req-5991592e-9389-476d-89a7-5720d017432a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:48.020685 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2aad0f7b1749478a3824ef4c30f370', 'request_id': 'req-5991592e-9389-476d-89a7-5720d017432a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:48.023663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] The token cache is being invalidate because project cea2433debb24b698dd2836acd25996e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:48.023969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:48.024764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-c60214ba-bc07-4f39-a74f-48209adc24a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:48.030757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1199/2387] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:47 2022] DELETE /identity/v3/projects/cea2433debb24b698dd2836acd25996e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:48.033315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c7ea2-c746-44fd-811c-1f82e725413a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.033631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c7ea2-c746-44fd-811c-1f82e725413a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.034038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c7ea2-c746-44fd-811c-1f82e725413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.037212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] The token cache is being invalidate because project ca2aad0f7b1749478a3824ef4c30f370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:48.037609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:48.038377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5991592e-9389-476d-89a7-5720d017432a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:48.040107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1189/2388] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:47 2022] DELETE /identity/v3/projects/ca2aad0f7b1749478a3824ef4c30f370 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:48.045243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b39ca-fde3-4ff9-962e-3b8c10a1642d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.045731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b39ca-fde3-4ff9-962e-3b8c10a1642d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.046004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b39ca-fde3-4ff9-962e-3b8c10a1642d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.058726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8c3c7ea2-c746-44fd-811c-1f82e725413a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:48.060227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8c3c7ea2-c746-44fd-811c-1f82e725413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:48.071014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5b7b39ca-fde3-4ff9-962e-3b8c10a1642d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:48.072237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5b7b39ca-fde3-4ff9-962e-3b8c10a1642d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:48.138279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1200/2389] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:48.146261 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a3ea5-a799-4ec1-9cdf-18141f4d50c1 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.146525 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a3ea5-a799-4ec1-9cdf-18141f4d50c1 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.146796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a3ea5-a799-4ec1-9cdf-18141f4d50c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.161488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1190/2390] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:48.166597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-452a3ea5-a799-4ec1-9cdf-18141f4d50c1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:48.167755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-452a3ea5-a799-4ec1-9cdf-18141f4d50c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:48.170297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2295-91fc-49da-b637-7cd567c9b864 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.170587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2295-91fc-49da-b637-7cd567c9b864 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.170866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2295-91fc-49da-b637-7cd567c9b864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.177519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1201/2391] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:48.181473 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a3ea5-a799-4ec1-9cdf-18141f4d50c1 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.187006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:48.192473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d80e2295-91fc-49da-b637-7cd567c9b864 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:48.192674 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a3ea5-a799-4ec1-9cdf-18141f4d50c1 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.193786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d80e2295-91fc-49da-b637-7cd567c9b864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:48.198569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.199167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.199441 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.199709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.200352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:48.200687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.202286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.203241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1191/2392] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:48.207326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e2295-91fc-49da-b637-7cd567c9b864 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.215377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1202/2393] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:25:48.216364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e2295-91fc-49da-b637-7cd567c9b864 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.219806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.221154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.221747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.222053 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.222326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.222968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:48.223270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.224882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.230142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1f678-958f-42fb-a5af-80f9531342aa admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.235386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5bd19-ec4c-4952-a260-9d342f3c0add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.235969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5bd19-ec4c-4952-a260-9d342f3c0add admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.236247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5bd19-ec4c-4952-a260-9d342f3c0add admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.236517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5bd19-ec4c-4952-a260-9d342f3c0add admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.237172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5bd19-ec4c-4952-a260-9d342f3c0add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.240698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1192/2394] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:25:48.240828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5bd19-ec4c-4952-a260-9d342f3c0add admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.241586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.251993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d91d0-0876-425d-9c9c-762141d1c103 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.257064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fe56d-beb0-4bfd-950f-c325f3dbe692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.257991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fe56d-beb0-4bfd-950f-c325f3dbe692 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.258314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fe56d-beb0-4bfd-950f-c325f3dbe692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.258737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fe56d-beb0-4bfd-950f-c325f3dbe692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.259547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fe56d-beb0-4bfd-950f-c325f3dbe692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.261818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fe56d-beb0-4bfd-950f-c325f3dbe692 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.267770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1203/2395] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:48.271940 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5bd19-ec4c-4952-a260-9d342f3c0add admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.295209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1193/2396] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:48.296188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fe56d-beb0-4bfd-950f-c325f3dbe692 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.341254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5bd19-ec4c-4952-a260-9d342f3c0add admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.358016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fe56d-beb0-4bfd-950f-c325f3dbe692 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.363784 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50efbb-315d-4a72-b930-76d421e1bd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.364389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50efbb-315d-4a72-b930-76d421e1bd95 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.364705 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50efbb-315d-4a72-b930-76d421e1bd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.364982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50efbb-315d-4a72-b930-76d421e1bd95 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.365613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50efbb-315d-4a72-b930-76d421e1bd95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.367545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50efbb-315d-4a72-b930-76d421e1bd95 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.379590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:48.394641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1194/2397] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:48.395377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50efbb-315d-4a72-b930-76d421e1bd95 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.401514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b445c70-61f3-4caf-b877-7bca4632c0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.402245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b445c70-61f3-4caf-b877-7bca4632c0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.402529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b445c70-61f3-4caf-b877-7bca4632c0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.402800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b445c70-61f3-4caf-b877-7bca4632c0d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.403429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b445c70-61f3-4caf-b877-7bca4632c0d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.405250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50efbb-315d-4a72-b930-76d421e1bd95 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.405402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b445c70-61f3-4caf-b877-7bca4632c0d8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.410991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0bcbc-70e9-4c8a-a5b7-a10fd6748f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.411562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0bcbc-70e9-4c8a-a5b7-a10fd6748f51 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.411845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0bcbc-70e9-4c8a-a5b7-a10fd6748f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.412119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0bcbc-70e9-4c8a-a5b7-a10fd6748f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.412795 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0bcbc-70e9-4c8a-a5b7-a10fd6748f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.414389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0bcbc-70e9-4c8a-a5b7-a10fd6748f51 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.426621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1195/2398] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:48.430567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0bcbc-70e9-4c8a-a5b7-a10fd6748f51 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.432636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1204/2399] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:48.434222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b445c70-61f3-4caf-b877-7bca4632c0d8 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.440903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0bcbc-70e9-4c8a-a5b7-a10fd6748f51 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.444291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b445c70-61f3-4caf-b877-7bca4632c0d8 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.445772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b338da-826e-40ad-b70c-7e89e5966ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.446364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b338da-826e-40ad-b70c-7e89e5966ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.446648 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b338da-826e-40ad-b70c-7e89e5966ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.446921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b338da-826e-40ad-b70c-7e89e5966ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.447569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b338da-826e-40ad-b70c-7e89e5966ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.449101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae636105-d24b-4171-b1eb-55d07fcf401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.449253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b338da-826e-40ad-b70c-7e89e5966ebc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.449691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636105-d24b-4171-b1eb-55d07fcf401b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.449973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636105-d24b-4171-b1eb-55d07fcf401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.450252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636105-d24b-4171-b1eb-55d07fcf401b admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.450894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae636105-d24b-4171-b1eb-55d07fcf401b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.452605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae636105-d24b-4171-b1eb-55d07fcf401b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.460277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1196/2400] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:48.465781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1205/2401] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:48.465921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b338da-826e-40ad-b70c-7e89e5966ebc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.466725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae636105-d24b-4171-b1eb-55d07fcf401b admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.475104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b338da-826e-40ad-b70c-7e89e5966ebc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.482047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae636105-d24b-4171-b1eb-55d07fcf401b admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.482201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0e7f5-b36d-46b8-a32e-9e2c203d4a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.483440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0e7f5-b36d-46b8-a32e-9e2c203d4a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.483998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0e7f5-b36d-46b8-a32e-9e2c203d4a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.485168 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0e7f5-b36d-46b8-a32e-9e2c203d4a9d admin admin] PATH_INFO: `/v3/projects/6e3fb544655b4cfc9f9789ccb190f44e/users/7cf8024e7d2647dfaabe0ef0694b6885/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.492433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80322-a8e1-4ef0-9ba5-bca8e090f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.493208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80322-a8e1-4ef0-9ba5-bca8e090f851 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.493491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80322-a8e1-4ef0-9ba5-bca8e090f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.493793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80322-a8e1-4ef0-9ba5-bca8e090f851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.494440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80322-a8e1-4ef0-9ba5-bca8e090f851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.496049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80322-a8e1-4ef0-9ba5-bca8e090f851 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.504222 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0e7f5-b36d-46b8-a32e-9e2c203d4a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3fb544655b4cfc9f9789ccb190f44e, user_id=7cf8024e7d2647dfaabe0ef0694b6885, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.506913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0e7f5-b36d-46b8-a32e-9e2c203d4a9d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.507585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1206/2402] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:48.511795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80322-a8e1-4ef0-9ba5-bca8e090f851 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.521095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1197/2403] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:48 2022] PUT /identity/v3/projects/6e3fb544655b4cfc9f9789ccb190f44e/users/7cf8024e7d2647dfaabe0ef0694b6885/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:48.524041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80322-a8e1-4ef0-9ba5-bca8e090f851 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.532192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0bb0-9976-42f8-8f9c-3e550d98d536 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.532479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0bb0-9976-42f8-8f9c-3e550d98d536 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.532858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0bb0-9976-42f8-8f9c-3e550d98d536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.562858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-625d0bb0-9976-42f8-8f9c-3e550d98d536 None None] MFA Rules not processed for user `7cf8024e7d2647dfaabe0ef0694b6885`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:48.564178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-625d0bb0-9976-42f8-8f9c-3e550d98d536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:48.570183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:48.586475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b14f0b-d473-46ac-a6cc-2182f589e88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.587163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14f0b-d473-46ac-a6cc-2182f589e88f admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.587440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14f0b-d473-46ac-a6cc-2182f589e88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.587712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14f0b-d473-46ac-a6cc-2182f589e88f admin admin] PATH_INFO: `/v3/projects/d5bd468566364f12abbacb3928316c78/users/bc1911f5d10c4fe38b6a6c4a0950726a/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.603339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b14f0b-d473-46ac-a6cc-2182f589e88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5bd468566364f12abbacb3928316c78, user_id=bc1911f5d10c4fe38b6a6c4a0950726a, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.606036 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b14f0b-d473-46ac-a6cc-2182f589e88f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.619686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1207/2404] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:48 2022] PUT /identity/v3/projects/d5bd468566364f12abbacb3928316c78/users/bc1911f5d10c4fe38b6a6c4a0950726a/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:48.624823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd15560-e3ca-4b3a-af57-94c9e2a83324 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.625145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd15560-e3ca-4b3a-af57-94c9e2a83324 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.625587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd15560-e3ca-4b3a-af57-94c9e2a83324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.632241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1198/2405] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:48.636676 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d0bb0-9976-42f8-8f9c-3e550d98d536 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.644327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fbd15560-e3ca-4b3a-af57-94c9e2a83324 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:48.645584 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fbd15560-e3ca-4b3a-af57-94c9e2a83324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:48.647816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d0bb0-9976-42f8-8f9c-3e550d98d536 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.704352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1208/2406] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:48.707380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3969d74-de0b-4221-9367-14e050eecaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.708125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3969d74-de0b-4221-9367-14e050eecaeb admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.708497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3969d74-de0b-4221-9367-14e050eecaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.708854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3969d74-de0b-4221-9367-14e050eecaeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.709552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3969d74-de0b-4221-9367-14e050eecaeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.711534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3969d74-de0b-4221-9367-14e050eecaeb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.713846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd15560-e3ca-4b3a-af57-94c9e2a83324 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.715193 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fbd15560-e3ca-4b3a-af57-94c9e2a83324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:48.742332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1199/2407] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:48.746181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3969d74-de0b-4221-9367-14e050eecaeb admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.774274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:48.803550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd15560-e3ca-4b3a-af57-94c9e2a83324 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.812530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3969d74-de0b-4221-9367-14e050eecaeb admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.817847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40378477-2870-43e8-bc9d-d7dc7425dea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.818637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378477-2870-43e8-bc9d-d7dc7425dea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.818912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378477-2870-43e8-bc9d-d7dc7425dea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.819318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40378477-2870-43e8-bc9d-d7dc7425dea4 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.819950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40378477-2870-43e8-bc9d-d7dc7425dea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.821909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40378477-2870-43e8-bc9d-d7dc7425dea4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.848755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1200/2408] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:48 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:48.849814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40378477-2870-43e8-bc9d-d7dc7425dea4 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.859637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40378477-2870-43e8-bc9d-d7dc7425dea4 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.864554 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7185b-6dcc-4993-b467-50a895f875d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.865343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7185b-6dcc-4993-b467-50a895f875d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.865569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89412457-0de2-4c14-bc3c-f0859d1e96c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.865694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7185b-6dcc-4993-b467-50a895f875d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.865886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7185b-6dcc-4993-b467-50a895f875d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.866179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412457-0de2-4c14-bc3c-f0859d1e96c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.866458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412457-0de2-4c14-bc3c-f0859d1e96c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.866591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7185b-6dcc-4993-b467-50a895f875d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.866722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412457-0de2-4c14-bc3c-f0859d1e96c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.868133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7185b-6dcc-4993-b467-50a895f875d9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.876143 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89412457-0de2-4c14-bc3c-f0859d1e96c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.877637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89412457-0de2-4c14-bc3c-f0859d1e96c7 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.879681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1201/2409] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:48.883659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7185b-6dcc-4993-b467-50a895f875d9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.889323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1209/2410] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:25:48.894961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7185b-6dcc-4993-b467-50a895f875d9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.899993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa102318-f902-48b5-875a-7fda857066e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:48.900751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa102318-f902-48b5-875a-7fda857066e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:48.901033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa102318-f902-48b5-875a-7fda857066e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:48.901309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa102318-f902-48b5-875a-7fda857066e0 admin admin] PATH_INFO: `/v3/projects/098af6ce5592404782d433ca819013bf/users/fa294413107142afa2bc4a64bfd7c768/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:48.919441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa102318-f902-48b5-875a-7fda857066e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=098af6ce5592404782d433ca819013bf, user_id=fa294413107142afa2bc4a64bfd7c768, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:48.922163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa102318-f902-48b5-875a-7fda857066e0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:48.936171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1202/2411] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:48 2022] PUT /identity/v3/projects/098af6ce5592404782d433ca819013bf/users/fa294413107142afa2bc4a64bfd7c768/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:48.940631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89412457-0de2-4c14-bc3c-f0859d1e96c7 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:48.950455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89412457-0de2-4c14-bc3c-f0859d1e96c7 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:48.955326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:49.020482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ceab06-65e2-42a7-9983-3207ceb81ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:49.021154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ceab06-65e2-42a7-9983-3207ceb81ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.021436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ceab06-65e2-42a7-9983-3207ceb81ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.021707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ceab06-65e2-42a7-9983-3207ceb81ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.022389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ceab06-65e2-42a7-9983-3207ceb81ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:49.024082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ceab06-65e2-42a7-9983-3207ceb81ac6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:49.035206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1210/2412] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:49.039667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa102318-f902-48b5-875a-7fda857066e0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:49.051640 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa102318-f902-48b5-875a-7fda857066e0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:49.056426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:49.057112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.057395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.057671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] PATH_INFO: `/v3/projects/098af6ce5592404782d433ca819013bf/users/fa294413107142afa2bc4a64bfd7c768/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.059629 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=098af6ce5592404782d433ca819013bf, user_id=fa294413107142afa2bc4a64bfd7c768, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:49.062362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:49.076475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1203/2413] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:49 2022] PUT /identity/v3/projects/098af6ce5592404782d433ca819013bf/users/fa294413107142afa2bc4a64bfd7c768/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:49.090091 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f782-fe98-4ae5-8b21-6e4a13097970 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.090383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f782-fe98-4ae5-8b21-6e4a13097970 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.090695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f782-fe98-4ae5-8b21-6e4a13097970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.133234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8ea6f782-fe98-4ae5-8b21-6e4a13097970 None None] MFA Rules not processed for user `fa294413107142afa2bc4a64bfd7c768`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:49.134571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8ea6f782-fe98-4ae5-8b21-6e4a13097970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:49.243809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1211/2414] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:49.254628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:49.257087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:49.349435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317bb22-0d94-4bb4-87cc-efd119e887e1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:49.349435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86946972-c0ab-46e3-b708-f467219a70fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:49.349435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946972-c0ab-46e3-b708-f467219a70fc service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.349435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946972-c0ab-46e3-b708-f467219a70fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.349952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946972-c0ab-46e3-b708-f467219a70fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.359076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86946972-c0ab-46e3-b708-f467219a70fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:49.361371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86946972-c0ab-46e3-b708-f467219a70fc service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:49.371941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1204/2415] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:49.513964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6f782-fe98-4ae5-8b21-6e4a13097970 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:49.595426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6f782-fe98-4ae5-8b21-6e4a13097970 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:49.600939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:49.601583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.601861 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.602132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] PATH_INFO: `/v3/users/7cf8024e7d2647dfaabe0ef0694b6885` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.603336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf8024e7d2647dfaabe0ef0694b6885)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:49.605357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:49.651602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:49.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf8024e7d2647dfaabe0ef0694b6885'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:49.680346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf8024e7d2647dfaabe0ef0694b6885'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:49.681281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf8024e7d2647dfaabe0ef0694b6885'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:49.708002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf8024e7d2647dfaabe0ef0694b6885'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:49.709000 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf8024e7d2647dfaabe0ef0694b6885'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:49.726980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1212/2416] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:49 2022] DELETE /identity/v3/users/7cf8024e7d2647dfaabe0ef0694b6885 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:49.731987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272df9-659a-4b6b-8017-31eb2b4e61bd None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.732341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272df9-659a-4b6b-8017-31eb2b4e61bd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.732544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272df9-659a-4b6b-8017-31eb2b4e61bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.755147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b9272df9-659a-4b6b-8017-31eb2b4e61bd None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:49.756634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b9272df9-659a-4b6b-8017-31eb2b4e61bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:49.827208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1205/2417] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:49.839205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:49.840807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:49.909693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56932-c82b-410c-9db3-7027d8fe71a0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:49.914503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53168bd6-9bf1-4c0a-bcea-1d02794393a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:49.915333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53168bd6-9bf1-4c0a-bcea-1d02794393a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:49.915768 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53168bd6-9bf1-4c0a-bcea-1d02794393a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:49.916953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53168bd6-9bf1-4c0a-bcea-1d02794393a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:49.928737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53168bd6-9bf1-4c0a-bcea-1d02794393a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:49.930424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53168bd6-9bf1-4c0a-bcea-1d02794393a4 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:49.942599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1213/2418] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:50.287882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272df9-659a-4b6b-8017-31eb2b4e61bd None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:50.289442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b9272df9-659a-4b6b-8017-31eb2b4e61bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:50.308847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272df9-659a-4b6b-8017-31eb2b4e61bd None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:50.314448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:50.315161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:50.315444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:50.315713 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] PATH_INFO: `/v3/projects/6e3fb544655b4cfc9f9789ccb190f44e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:50.317291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3fb544655b4cfc9f9789ccb190f44e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:50.319013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:50.321565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:50.395010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3fb544655b4cfc9f9789ccb190f44e', 'request_id': 'req-09c47198-14aa-4227-ad0f-b5901fe792ba'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.405860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3fb544655b4cfc9f9789ccb190f44e', 'request_id': 'req-09c47198-14aa-4227-ad0f-b5901fe792ba'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.415682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3fb544655b4cfc9f9789ccb190f44e', 'request_id': 'req-09c47198-14aa-4227-ad0f-b5901fe792ba'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.431108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] The token cache is being invalidate because project 6e3fb544655b4cfc9f9789ccb190f44e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:50.431404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.432215 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.434299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1206/2419] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:50 2022] DELETE /identity/v3/projects/6e3fb544655b4cfc9f9789ccb190f44e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:50.439400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53168bd6-9bf1-4c0a-bcea-1d02794393a4 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:50.479121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53168bd6-9bf1-4c0a-bcea-1d02794393a4 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:50.596617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:50.597234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:50.597510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:50.597943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] PATH_INFO: `/v3/users/fa294413107142afa2bc4a64bfd7c768` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:50.599292 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa294413107142afa2bc4a64bfd7c768)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:50.601080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:50.657964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa294413107142afa2bc4a64bfd7c768'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.663425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:50.674327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa294413107142afa2bc4a64bfd7c768'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.675613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa294413107142afa2bc4a64bfd7c768'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.706307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa294413107142afa2bc4a64bfd7c768'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.707927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4fe1e559-2579-4d5c-a449-439a9431900f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa294413107142afa2bc4a64bfd7c768'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:50.735570 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1214/2420] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:50 2022] DELETE /identity/v3/users/fa294413107142afa2bc4a64bfd7c768 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:51.165746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:51.167799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:51.182623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47198-14aa-4227-ad0f-b5901fe792ba admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:51.259140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:51.259789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.260068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.260343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] PATH_INFO: `/v3/projects/098af6ce5592404782d433ca819013bf` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:51.261522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] RBAC: Authorizing `identity:delete_project(project_id=098af6ce5592404782d433ca819013bf)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:51.263514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:51.340387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098af6ce5592404782d433ca819013bf', 'request_id': 'req-0c383778-0209-4388-9470-3f88d8ea40af'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:51.363699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098af6ce5592404782d433ca819013bf', 'request_id': 'req-0c383778-0209-4388-9470-3f88d8ea40af'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:51.371071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098af6ce5592404782d433ca819013bf', 'request_id': 'req-0c383778-0209-4388-9470-3f88d8ea40af'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:51.386928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] The token cache is being invalidate because project 098af6ce5592404782d433ca819013bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:51.387415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:51.388191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0c383778-0209-4388-9470-3f88d8ea40af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:51.393262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1207/2421] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:51 2022] DELETE /identity/v3/projects/098af6ce5592404782d433ca819013bf => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:51.395247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfdb26-3742-4394-8dd1-b514e7179063 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.395722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfdb26-3742-4394-8dd1-b514e7179063 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.395985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfdb26-3742-4394-8dd1-b514e7179063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:51.419173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6bcfdb26-3742-4394-8dd1-b514e7179063 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:51.420625 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6bcfdb26-3742-4394-8dd1-b514e7179063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:51.493518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1215/2422] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:51.501609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf9dc-3eae-4ac3-9e92-461f4af2b822 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.502049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf9dc-3eae-4ac3-9e92-461f4af2b822 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.502422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf9dc-3eae-4ac3-9e92-461f4af2b822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:51.525533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b4fcf9dc-3eae-4ac3-9e92-461f4af2b822 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:51.526851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b4fcf9dc-3eae-4ac3-9e92-461f4af2b822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:51.540756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1208/2423] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:51.542101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfdb26-3742-4394-8dd1-b514e7179063 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:51.554239 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfdb26-3742-4394-8dd1-b514e7179063 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:51.559918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:51.560760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.561141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.561516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:51.562283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:51.562765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:51.564651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:51.580119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1216/2424] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:25:51.583824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf9dc-3eae-4ac3-9e92-461f4af2b822 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:51.597172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf9dc-3eae-4ac3-9e92-461f4af2b822 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:51.603402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9eb94d-75bc-4e39-8225-b69b6651a4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:51.604109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eb94d-75bc-4e39-8225-b69b6651a4db admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.604495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eb94d-75bc-4e39-8225-b69b6651a4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.605021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eb94d-75bc-4e39-8225-b69b6651a4db admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:51.606227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9eb94d-75bc-4e39-8225-b69b6651a4db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:51.610394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9eb94d-75bc-4e39-8225-b69b6651a4db admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:51.639012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:51.666756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1209/2425] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:51 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:51.670506 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:51.691267 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6e995-9aa2-41dc-a0a3-e48b5d69fa40 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:51.816595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140e8a8-1445-4877-ae4f-a27a2b4823eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:51.817135 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140e8a8-1445-4877-ae4f-a27a2b4823eb admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.817420 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140e8a8-1445-4877-ae4f-a27a2b4823eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.817698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140e8a8-1445-4877-ae4f-a27a2b4823eb admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:51.818357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140e8a8-1445-4877-ae4f-a27a2b4823eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:51.820480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140e8a8-1445-4877-ae4f-a27a2b4823eb admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:51.876959 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1217/2426] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:51 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:51.881415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9eb94d-75bc-4e39-8225-b69b6651a4db admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:51.893045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9eb94d-75bc-4e39-8225-b69b6651a4db admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:51.898048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d8abc-535f-452c-89c4-ae58b1530348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:51.898670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d8abc-535f-452c-89c4-ae58b1530348 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.899431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d8abc-535f-452c-89c4-ae58b1530348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.899575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d8abc-535f-452c-89c4-ae58b1530348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:51.900674 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d8abc-535f-452c-89c4-ae58b1530348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:51.902325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d8abc-535f-452c-89c4-ae58b1530348 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:51.920702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1210/2427] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:51.922169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140e8a8-1445-4877-ae4f-a27a2b4823eb admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:51.934415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140e8a8-1445-4877-ae4f-a27a2b4823eb admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:51.941567 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a191c-1f67-404f-9774-a2f30c27533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:51.943310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a191c-1f67-404f-9774-a2f30c27533d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.943605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a191c-1f67-404f-9774-a2f30c27533d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.943880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a191c-1f67-404f-9774-a2f30c27533d admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:51.945475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a191c-1f67-404f-9774-a2f30c27533d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:51.949134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a191c-1f67-404f-9774-a2f30c27533d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:51.966360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1218/2428] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:51.969512 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d8abc-535f-452c-89c4-ae58b1530348 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:51.980779 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d8abc-535f-452c-89c4-ae58b1530348 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:51.985510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd5a35-3091-4e7a-b47f-9cdc1dcad175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:51.986178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd5a35-3091-4e7a-b47f-9cdc1dcad175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:51.987034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd5a35-3091-4e7a-b47f-9cdc1dcad175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:51.987312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd5a35-3091-4e7a-b47f-9cdc1dcad175 admin admin] PATH_INFO: `/v3/projects/467228e44f4a4e6c86d42ba43a9decf9/users/fc57516c8b874b809abefd19e6695fdb/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.006143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd5a35-3091-4e7a-b47f-9cdc1dcad175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467228e44f4a4e6c86d42ba43a9decf9, user_id=fc57516c8b874b809abefd19e6695fdb, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:52.008750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd5a35-3091-4e7a-b47f-9cdc1dcad175 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:52.026579 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1211/2429] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:51 2022] PUT /identity/v3/projects/467228e44f4a4e6c86d42ba43a9decf9/users/fc57516c8b874b809abefd19e6695fdb/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:52.039287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310113f8-e09b-479d-93a0-52037e52e1d3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:52.040048 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310113f8-e09b-479d-93a0-52037e52e1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:52.040348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310113f8-e09b-479d-93a0-52037e52e1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.089105 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-310113f8-e09b-479d-93a0-52037e52e1d3 None None] MFA Rules not processed for user `fc57516c8b874b809abefd19e6695fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:52.090512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-310113f8-e09b-479d-93a0-52037e52e1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:52.119090 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:52.188482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1219/2430] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:52.193107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd5a35-3091-4e7a-b47f-9cdc1dcad175 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:52.273328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd5a35-3091-4e7a-b47f-9cdc1dcad175 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:52.283357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373911-a47d-4d4c-87c4-83cefcb68173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:52.284109 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373911-a47d-4d4c-87c4-83cefcb68173 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:52.284677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373911-a47d-4d4c-87c4-83cefcb68173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:52.285025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92373911-a47d-4d4c-87c4-83cefcb68173 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.285756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92373911-a47d-4d4c-87c4-83cefcb68173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:52.289838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92373911-a47d-4d4c-87c4-83cefcb68173 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:52.328143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1212/2431] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:52 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:52.333060 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310113f8-e09b-479d-93a0-52037e52e1d3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:52.348936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310113f8-e09b-479d-93a0-52037e52e1d3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:52.428647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b99c82-b2cb-4cdd-b247-85c4cded4f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:52.429515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b99c82-b2cb-4cdd-b247-85c4cded4f3d admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:52.429806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b99c82-b2cb-4cdd-b247-85c4cded4f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:52.430344 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b99c82-b2cb-4cdd-b247-85c4cded4f3d admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.431147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b99c82-b2cb-4cdd-b247-85c4cded4f3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:52.433382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b99c82-b2cb-4cdd-b247-85c4cded4f3d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:52.469443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373911-a47d-4d4c-87c4-83cefcb68173 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:52.469747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1220/2432] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:52 2022] POST /identity/v3/users => generated 298 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:52.482878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92373911-a47d-4d4c-87c4-83cefcb68173 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:52.489933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5b13d-3f0e-4f1b-af86-92edf42c0494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:52.491256 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5b13d-3f0e-4f1b-af86-92edf42c0494 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:52.491734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5b13d-3f0e-4f1b-af86-92edf42c0494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:52.492167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5b13d-3f0e-4f1b-af86-92edf42c0494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.493002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5b13d-3f0e-4f1b-af86-92edf42c0494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:52.494892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5b13d-3f0e-4f1b-af86-92edf42c0494 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:52.512387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1213/2433] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:52.512620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b99c82-b2cb-4cdd-b247-85c4cded4f3d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:52.527012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b99c82-b2cb-4cdd-b247-85c4cded4f3d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:52.534458 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342979-af82-4523-9d7c-85544cbb0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:52.534704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342979-af82-4523-9d7c-85544cbb0082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:52.534962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342979-af82-4523-9d7c-85544cbb0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:52.535305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342979-af82-4523-9d7c-85544cbb0082 admin admin] PATH_INFO: `/v3/projects/6010eb0da187447d9b99599a22449d02/users/0a37ed6b01174f958d89b84bd1c44386/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.555249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0342979-af82-4523-9d7c-85544cbb0082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6010eb0da187447d9b99599a22449d02, user_id=0a37ed6b01174f958d89b84bd1c44386, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:52.557787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0342979-af82-4523-9d7c-85544cbb0082 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:52.579566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1221/2434] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:52 2022] PUT /identity/v3/projects/6010eb0da187447d9b99599a22449d02/users/0a37ed6b01174f958d89b84bd1c44386/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:52.583360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5b13d-3f0e-4f1b-af86-92edf42c0494 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:52.595156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5b13d-3f0e-4f1b-af86-92edf42c0494 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:52.661361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57981c35-2cc7-4292-a2e9-ddafcd6bf465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:52.662073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57981c35-2cc7-4292-a2e9-ddafcd6bf465 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:52.662404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57981c35-2cc7-4292-a2e9-ddafcd6bf465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:52.662740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57981c35-2cc7-4292-a2e9-ddafcd6bf465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.663451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57981c35-2cc7-4292-a2e9-ddafcd6bf465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:52.665564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57981c35-2cc7-4292-a2e9-ddafcd6bf465 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:52.678324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1214/2435] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:52.684858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342979-af82-4523-9d7c-85544cbb0082 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:52.711972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342979-af82-4523-9d7c-85544cbb0082 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:52.727557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:52.728956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:52.732937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:52.733465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] PATH_INFO: `/v3/projects/6010eb0da187447d9b99599a22449d02/users/0a37ed6b01174f958d89b84bd1c44386/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.736561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6010eb0da187447d9b99599a22449d02, user_id=0a37ed6b01174f958d89b84bd1c44386, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:52.739317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:52.755219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1222/2436] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:52 2022] PUT /identity/v3/projects/6010eb0da187447d9b99599a22449d02/users/0a37ed6b01174f958d89b84bd1c44386/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:52.766513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813af64-3e09-4d81-baf3-4a4bee2e7c1a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:52.766884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813af64-3e09-4d81-baf3-4a4bee2e7c1a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:52.767070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813af64-3e09-4d81-baf3-4a4bee2e7c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:52.816366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2813af64-3e09-4d81-baf3-4a4bee2e7c1a None None] MFA Rules not processed for user `0a37ed6b01174f958d89b84bd1c44386`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:52.818290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2813af64-3e09-4d81-baf3-4a4bee2e7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:52.837244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:52.943229 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1215/2437] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:52.956865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:52.959747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:53.052092 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab07abb-c611-4d44-a184-322335dabf7a admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:53.075724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce8bf6-7508-4351-88ef-405cff925675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:53.076367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce8bf6-7508-4351-88ef-405cff925675 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:53.077363 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce8bf6-7508-4351-88ef-405cff925675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:53.077651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce8bf6-7508-4351-88ef-405cff925675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:53.092995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce8bf6-7508-4351-88ef-405cff925675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:53.095867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce8bf6-7508-4351-88ef-405cff925675 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:53.132977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1223/2438] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:53.222659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813af64-3e09-4d81-baf3-4a4bee2e7c1a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:53.232802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813af64-3e09-4d81-baf3-4a4bee2e7c1a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:53.237730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa835f6-d8a6-425f-af8e-7990f3141f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:53.238469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa835f6-d8a6-425f-af8e-7990f3141f23 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:53.238820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa835f6-d8a6-425f-af8e-7990f3141f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:53.239158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa835f6-d8a6-425f-af8e-7990f3141f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:53.312072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa835f6-d8a6-425f-af8e-7990f3141f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:53.313845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa835f6-d8a6-425f-af8e-7990f3141f23 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:53.326273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1216/2439] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:25:53.669689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e6b3e-705e-4cf6-9c8a-62243ad06f05 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:53.670278 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e6b3e-705e-4cf6-9c8a-62243ad06f05 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:53.670571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e6b3e-705e-4cf6-9c8a-62243ad06f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:53.715991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fe6e6b3e-705e-4cf6-9c8a-62243ad06f05 None None] MFA Rules not processed for user `bc1911f5d10c4fe38b6a6c4a0950726a`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:53.719279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fe6e6b3e-705e-4cf6-9c8a-62243ad06f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:53.736547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:53.753841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa835f6-d8a6-425f-af8e-7990f3141f23 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:53.823058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1224/2440] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:53.827488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e6b3e-705e-4cf6-9c8a-62243ad06f05 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:53.829824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fe6e6b3e-705e-4cf6-9c8a-62243ad06f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:53.884469 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa835f6-d8a6-425f-af8e-7990f3141f23 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:53.891211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:53.891861 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:53.892137 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:53.892405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] PATH_INFO: `/v3/users/fc57516c8b874b809abefd19e6695fdb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:53.893846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc57516c8b874b809abefd19e6695fdb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:53.898236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:53.965805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e6b3e-705e-4cf6-9c8a-62243ad06f05 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:53.972062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:53.973766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:53.974169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:53.974540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:53.975442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:53.976941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc57516c8b874b809abefd19e6695fdb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:53.977756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:53.983043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:53.998105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc57516c8b874b809abefd19e6695fdb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:53.998905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc57516c8b874b809abefd19e6695fdb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:53.999522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc57516c8b874b809abefd19e6695fdb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:54.013682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4d0f4f8-c0d5-482e-831c-84290144fc37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc57516c8b874b809abefd19e6695fdb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:54.051849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1225/2441] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:53 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:54.055968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1217/2442] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:53 2022] DELETE /identity/v3/users/fc57516c8b874b809abefd19e6695fdb => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:54.057073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:54.059174 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:54.060272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671fcd0-ca29-4cc9-9edc-3d1dbb35ba2e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.060604 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671fcd0-ca29-4cc9-9edc-3d1dbb35ba2e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.063367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671fcd0-ca29-4cc9-9edc-3d1dbb35ba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.097928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4671fcd0-ca29-4cc9-9edc-3d1dbb35ba2e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:54.099311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4671fcd0-ca29-4cc9-9edc-3d1dbb35ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:54.160864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f97fb-d28c-4756-bd83-04aed70494fa admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:54.190647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1218/2443] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:25:54.202012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671fcd0-ca29-4cc9-9edc-3d1dbb35ba2e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:54.204119 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4671fcd0-ca29-4cc9-9edc-3d1dbb35ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:54.219444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:54.245527 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9e3b7-b25a-49cb-bb96-b50d4b6baa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:54.246333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9e3b7-b25a-49cb-bb96-b50d4b6baa38 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.246724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9e3b7-b25a-49cb-bb96-b50d4b6baa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.247099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9e3b7-b25a-49cb-bb96-b50d4b6baa38 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.247967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9e3b7-b25a-49cb-bb96-b50d4b6baa38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:54.250228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9e3b7-b25a-49cb-bb96-b50d4b6baa38 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:54.278869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1226/2444] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:54 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:54.280079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9e3b7-b25a-49cb-bb96-b50d4b6baa38 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:54.290232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671fcd0-ca29-4cc9-9edc-3d1dbb35ba2e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:54.292408 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9e3b7-b25a-49cb-bb96-b50d4b6baa38 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:54.294244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0ee7a-7563-4a02-a122-a459d5b34d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:54.294865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0ee7a-7563-4a02-a122-a459d5b34d63 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.295143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0ee7a-7563-4a02-a122-a459d5b34d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.295413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0ee7a-7563-4a02-a122-a459d5b34d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.298193 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78952081-a3a4-4118-8f68-80edf33bf495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:54.298933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78952081-a3a4-4118-8f68-80edf33bf495 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.299282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78952081-a3a4-4118-8f68-80edf33bf495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.299622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78952081-a3a4-4118-8f68-80edf33bf495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.300391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78952081-a3a4-4118-8f68-80edf33bf495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:54.302294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78952081-a3a4-4118-8f68-80edf33bf495 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:54.305498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0ee7a-7563-4a02-a122-a459d5b34d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:54.306952 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0ee7a-7563-4a02-a122-a459d5b34d63 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:54.318211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1219/2445] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:25:54.319756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1227/2446] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:54.320529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0ee7a-7563-4a02-a122-a459d5b34d63 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:54.330892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0ee7a-7563-4a02-a122-a459d5b34d63 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:54.335762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b0a3d-e76f-4d0d-ae10-0634f17eb5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:54.336425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b0a3d-e76f-4d0d-ae10-0634f17eb5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.336740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b0a3d-e76f-4d0d-ae10-0634f17eb5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.337059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b0a3d-e76f-4d0d-ae10-0634f17eb5fa admin admin] PATH_INFO: `/v3/projects/c12ebc76295b4655924ebad82f1ed832/users/df02b66f2fbe41448d6c9f85a5e7e4fe/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.357827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b0a3d-e76f-4d0d-ae10-0634f17eb5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12ebc76295b4655924ebad82f1ed832, user_id=df02b66f2fbe41448d6c9f85a5e7e4fe, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:54.361806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b0a3d-e76f-4d0d-ae10-0634f17eb5fa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:54.394900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1220/2447] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:54 2022] PUT /identity/v3/projects/c12ebc76295b4655924ebad82f1ed832/users/df02b66f2fbe41448d6c9f85a5e7e4fe/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:54.399355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78952081-a3a4-4118-8f68-80edf33bf495 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:54.412224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78952081-a3a4-4118-8f68-80edf33bf495 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:54.494953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:54.495590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.495924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.496388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.497338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:54.499284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:54.511323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1228/2448] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:54.515474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b0a3d-e76f-4d0d-ae10-0634f17eb5fa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:54.530564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b0a3d-e76f-4d0d-ae10-0634f17eb5fa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:54.537589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:54.540641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.540641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.540641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] PATH_INFO: `/v3/projects/c12ebc76295b4655924ebad82f1ed832/users/df02b66f2fbe41448d6c9f85a5e7e4fe/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.546321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12ebc76295b4655924ebad82f1ed832, user_id=df02b66f2fbe41448d6c9f85a5e7e4fe, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:54.553009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:54.568228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1221/2449] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:54 2022] PUT /identity/v3/projects/c12ebc76295b4655924ebad82f1ed832/users/df02b66f2fbe41448d6c9f85a5e7e4fe/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:54.753032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:54.753552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:54.771720 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fdede8-d335-461c-8b80-859b44ab7894 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:54.891287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:54.892687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:54.893255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:54.894245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] PATH_INFO: `/v3/projects/467228e44f4a4e6c86d42ba43a9decf9` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:54.896969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] RBAC: Authorizing `identity:delete_project(project_id=467228e44f4a4e6c86d42ba43a9decf9)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:54.903102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:55.064053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467228e44f4a4e6c86d42ba43a9decf9', 'request_id': 'req-3fd7535e-1db2-4669-9071-c4e69541b65a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.073469 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467228e44f4a4e6c86d42ba43a9decf9', 'request_id': 'req-3fd7535e-1db2-4669-9071-c4e69541b65a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.080910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:55.087087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467228e44f4a4e6c86d42ba43a9decf9', 'request_id': 'req-3fd7535e-1db2-4669-9071-c4e69541b65a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.114125 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] The token cache is being invalidate because project 467228e44f4a4e6c86d42ba43a9decf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:55.114624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.115630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3fd7535e-1db2-4669-9071-c4e69541b65a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.130998 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1229/2450] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:54 2022] DELETE /identity/v3/projects/467228e44f4a4e6c86d42ba43a9decf9 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:25:55.133066 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:55.137915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:55.171327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b24039-a3f2-4e5a-b300-b94766013625 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:55.249487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:55.335728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:55.338178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:55.339922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:55.341669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] PATH_INFO: `/v3/users/0a37ed6b01174f958d89b84bd1c44386` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:55.344893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a37ed6b01174f958d89b84bd1c44386)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:55.352917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:55.469873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a37ed6b01174f958d89b84bd1c44386'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.510876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a37ed6b01174f958d89b84bd1c44386'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.512126 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a37ed6b01174f958d89b84bd1c44386'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.512949 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a37ed6b01174f958d89b84bd1c44386'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.536980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a37ed6b01174f958d89b84bd1c44386'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:55.559624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1222/2451] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:25:55 2022] DELETE /identity/v3/users/0a37ed6b01174f958d89b84bd1c44386 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:55.919081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782028a3-e6ae-42e2-b778-4e4d4fb430c3 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:55.919575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782028a3-e6ae-42e2-b778-4e4d4fb430c3 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:55.919687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782028a3-e6ae-42e2-b778-4e4d4fb430c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:55.953686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-782028a3-e6ae-42e2-b778-4e4d4fb430c3 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:55.955657 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-782028a3-e6ae-42e2-b778-4e4d4fb430c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:56.040097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1230/2452] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:25:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:56.073268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:56.075373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:56.089252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782028a3-e6ae-42e2-b778-4e4d4fb430c3 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:56.095137 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-782028a3-e6ae-42e2-b778-4e4d4fb430c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:56.241487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e30845-7cbb-4269-9d08-82569281f866 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:56.248344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffe90a-dc6c-4a8d-bd91-a073c369564b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:56.249058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffe90a-dc6c-4a8d-bd91-a073c369564b service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:56.249342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffe90a-dc6c-4a8d-bd91-a073c369564b service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:56.249617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffe90a-dc6c-4a8d-bd91-a073c369564b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:56.260091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffe90a-dc6c-4a8d-bd91-a073c369564b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:56.262343 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffe90a-dc6c-4a8d-bd91-a073c369564b service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:56.265103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782028a3-e6ae-42e2-b778-4e4d4fb430c3 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:56.274019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:56.275111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1223/2453] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:25:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:25:56.284596 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:56.285204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:56.285874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] PATH_INFO: `/v3/projects/6010eb0da187447d9b99599a22449d02` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:56.288623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6010eb0da187447d9b99599a22449d02)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:56.291679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:56.387948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6010eb0da187447d9b99599a22449d02', 'request_id': 'req-61c78dbc-3897-4191-8a2b-0759c8db1917'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:56.400368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6010eb0da187447d9b99599a22449d02', 'request_id': 'req-61c78dbc-3897-4191-8a2b-0759c8db1917'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:56.410821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6010eb0da187447d9b99599a22449d02', 'request_id': 'req-61c78dbc-3897-4191-8a2b-0759c8db1917'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:56.442169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] The token cache is being invalidate because project 6010eb0da187447d9b99599a22449d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:25:56.443064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:56.443839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-61c78dbc-3897-4191-8a2b-0759c8db1917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:25:56.449050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1231/2454] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:25:56 2022] DELETE /identity/v3/projects/6010eb0da187447d9b99599a22449d02 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:56.451104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c7570-f4aa-468d-a9e1-302d9a33445d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:56.452227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c7570-f4aa-468d-a9e1-302d9a33445d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:56.452780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c7570-f4aa-468d-a9e1-302d9a33445d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:56.482937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c52c7570-f4aa-468d-a9e1-302d9a33445d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:56.486045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c52c7570-f4aa-468d-a9e1-302d9a33445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:56.550802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:56.613829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1224/2455] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:56.616772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677e16-4f85-499f-ab6a-155cafff2482 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:56.617100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677e16-4f85-499f-ab6a-155cafff2482 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:56.617378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5677e16-4f85-499f-ab6a-155cafff2482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:56.670729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-d5677e16-4f85-499f-ab6a-155cafff2482 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:56.673191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d5677e16-4f85-499f-ab6a-155cafff2482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:56.698513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1232/2456] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:56.699691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c7570-f4aa-468d-a9e1-302d9a33445d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:56.714968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c7570-f4aa-468d-a9e1-302d9a33445d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:56.728871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:56.730150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:56.730668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:56.731654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:56.732562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:25:56.737706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:56.739811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:56.769817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1225/2457] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:25:56.774281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5677e16-4f85-499f-ab6a-155cafff2482 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:56.795645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5677e16-4f85-499f-ab6a-155cafff2482 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:56.834096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf361ab-4d5b-449c-b3c2-80fc9449afc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:56.837424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf361ab-4d5b-449c-b3c2-80fc9449afc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:56.837995 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf361ab-4d5b-449c-b3c2-80fc9449afc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:56.838485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf361ab-4d5b-449c-b3c2-80fc9449afc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:56.839503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf361ab-4d5b-449c-b3c2-80fc9449afc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:56.842891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf361ab-4d5b-449c-b3c2-80fc9449afc7 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:56.878120 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:25:56.915011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1233/2458] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:25:56 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:25:56.915438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:56.927535 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaca17-0cf2-426a-9ed9-da447bd906c3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:57.067976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8a777-72f5-4148-80b5-10e7769fdf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:57.068730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8a777-72f5-4148-80b5-10e7769fdf77 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:57.069085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8a777-72f5-4148-80b5-10e7769fdf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:57.069424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8a777-72f5-4148-80b5-10e7769fdf77 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:57.070149 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8a777-72f5-4148-80b5-10e7769fdf77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:57.072210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8a777-72f5-4148-80b5-10e7769fdf77 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:57.128227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1226/2459] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:25:56 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:25:57.132274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf361ab-4d5b-449c-b3c2-80fc9449afc7 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:57.152718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf361ab-4d5b-449c-b3c2-80fc9449afc7 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:57.173074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a31b14-4ec9-40cd-93b1-616c9323d816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:57.174370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31b14-4ec9-40cd-93b1-616c9323d816 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:57.174657 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31b14-4ec9-40cd-93b1-616c9323d816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:57.177645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31b14-4ec9-40cd-93b1-616c9323d816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:57.178367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a31b14-4ec9-40cd-93b1-616c9323d816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:57.184023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a31b14-4ec9-40cd-93b1-616c9323d816 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:57.221003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8a777-72f5-4148-80b5-10e7769fdf77 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:57.223173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1234/2460] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:25:57.235786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8a777-72f5-4148-80b5-10e7769fdf77 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:57.248651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192439b8-bf91-4f14-8ecb-97a7076143a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:57.249737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192439b8-bf91-4f14-8ecb-97a7076143a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:57.250937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192439b8-bf91-4f14-8ecb-97a7076143a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:57.251451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192439b8-bf91-4f14-8ecb-97a7076143a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:57.253613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192439b8-bf91-4f14-8ecb-97a7076143a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:57.255916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192439b8-bf91-4f14-8ecb-97a7076143a7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:57.271057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1227/2461] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:25:57.276096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a31b14-4ec9-40cd-93b1-616c9323d816 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:57.289265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a31b14-4ec9-40cd-93b1-616c9323d816 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:57.295948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:57.296988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:57.297717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:57.297817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] PATH_INFO: `/v3/projects/555935a811b447698269617fca8e7f6c/users/3f434638fb384c408fcf04690c368b48/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:57.322834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=555935a811b447698269617fca8e7f6c, user_id=3f434638fb384c408fcf04690c368b48, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:57.327391 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:57.363383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1235/2462] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:25:57 2022] PUT /identity/v3/projects/555935a811b447698269617fca8e7f6c/users/3f434638fb384c408fcf04690c368b48/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:25:57.374612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed084-ec7e-4a5c-a005-f079678da142 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:57.376863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed084-ec7e-4a5c-a005-f079678da142 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:57.377365 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed084-ec7e-4a5c-a005-f079678da142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:57.455856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a76ed084-ec7e-4a5c-a005-f079678da142 None None] MFA Rules not processed for user `3f434638fb384c408fcf04690c368b48`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:25:57.460414 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a76ed084-ec7e-4a5c-a005-f079678da142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:57.673235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1228/2463] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:25:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:25:57.684485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:25:57.686798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:25:57.858840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a902c-008c-4864-94e5-4a7fd120f404 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:25:57.869083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6c90a2b95c498494a73b43e7138905', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:25:57.869719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:25:57.872075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:25:57.872367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:25:57.891364 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:25:57.893340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:25:57.908681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1236/2464] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:02.367672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbdf06-e08b-43ab-b602-7bfd170daa2b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:02.368290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbdf06-e08b-43ab-b602-7bfd170daa2b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:02.368290 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbdf06-e08b-43ab-b602-7bfd170daa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:02.429561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d8bbdf06-e08b-43ab-b602-7bfd170daa2b None None] MFA Rules not processed for user `df02b66f2fbe41448d6c9f85a5e7e4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:02.430549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d8bbdf06-e08b-43ab-b602-7bfd170daa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:02.473737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:02.556963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1229/2465] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:02.571673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:02.574396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:02.730936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f022af-1146-4868-aab8-5441f41a65d7 service glance] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:02.740782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8679a-4a2f-4b5b-a073-65a8265dc9ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:02.741418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8679a-4a2f-4b5b-a073-65a8265dc9ed service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:02.741895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8679a-4a2f-4b5b-a073-65a8265dc9ed service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:02.742004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8679a-4a2f-4b5b-a073-65a8265dc9ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:02.743841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-32a8679a-4a2f-4b5b-a073-65a8265dc9ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:02.835037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:02.871648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8679a-4a2f-4b5b-a073-65a8265dc9ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:02.876812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8679a-4a2f-4b5b-a073-65a8265dc9ed service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:02.887851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1237/2466] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:12.551172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=15 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:12.884093 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=10 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:18.003653 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbdf06-e08b-43ab-b602-7bfd170daa2b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:18.005651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d8bbdf06-e08b-43ab-b602-7bfd170daa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:18.076728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbdf06-e08b-43ab-b602-7bfd170daa2b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:18.082224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778705d-839f-4492-920d-481d421b6d4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:18.082827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778705d-839f-4492-920d-481d421b6d4c service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:18.083102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778705d-839f-4492-920d-481d421b6d4c service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:18.083371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778705d-839f-4492-920d-481d421b6d4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:18.085804 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d778705d-839f-4492-920d-481d421b6d4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:18.097983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778705d-839f-4492-920d-481d421b6d4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:18.099577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778705d-839f-4492-920d-481d421b6d4c service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:18.110090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1230/2467] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:20.779396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9d611-41ae-47be-a9fc-023657cfc250 None None] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:20.780872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9d611-41ae-47be-a9fc-023657cfc250 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:20.781497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9d611-41ae-47be-a9fc-023657cfc250 None None] PATH_INFO: `/` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:20.783236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1238/2468] 10.222.0.95 () {54 vars in 921 bytes} [Wed Aug 31 19:26:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:26:20.789051 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e0ef-9f76-45f1-a710-d0b3d12ad624 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:20.789599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e0ef-9f76-45f1-a710-d0b3d12ad624 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:20.789599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e0ef-9f76-45f1-a710-d0b3d12ad624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:20.810930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5042e0ef-9f76-45f1-a710-d0b3d12ad624 None None] MFA Rules not processed for user `aaab594854824542bd85426b0f16cbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:20.813162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5042e0ef-9f76-45f1-a710-d0b3d12ad624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:20.832799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1231/2469] 10.222.0.95 () {58 vars in 1018 bytes} [Wed Aug 31 19:26:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:20.844387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9d611-41ae-47be-a9fc-023657cfc250 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:20.857068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9d611-41ae-47be-a9fc-023657cfc250 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:20.861021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b782c-f96c-42bf-bb2b-89f3f96fd434 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6c90a2b95c498494a73b43e7138905', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:20.861712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b782c-f96c-42bf-bb2b-89f3f96fd434 service glance] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:20.862097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b782c-f96c-42bf-bb2b-89f3f96fd434 service glance] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:20.862369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b782c-f96c-42bf-bb2b-89f3f96fd434 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:20.877417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b782c-f96c-42bf-bb2b-89f3f96fd434 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:20.878896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b782c-f96c-42bf-bb2b-89f3f96fd434 service glance] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:20.891128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1239/2470] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:21.038553 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042e0ef-9f76-45f1-a710-d0b3d12ad624 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:21.039957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5042e0ef-9f76-45f1-a710-d0b3d12ad624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:21.099449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042e0ef-9f76-45f1-a710-d0b3d12ad624 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:21.103130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a7b1e-0c25-4e12-97f5-92ba03211f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:21.103711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a7b1e-0c25-4e12-97f5-92ba03211f72 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:21.103992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a7b1e-0c25-4e12-97f5-92ba03211f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:21.104262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a7b1e-0c25-4e12-97f5-92ba03211f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:21.106048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-509a7b1e-0c25-4e12-97f5-92ba03211f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:21.120417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a7b1e-0c25-4e12-97f5-92ba03211f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:21.122197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a7b1e-0c25-4e12-97f5-92ba03211f72 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:21.136483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1232/2471] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:23.083735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b782c-f96c-42bf-bb2b-89f3f96fd434 service glance] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:23.156677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b782c-f96c-42bf-bb2b-89f3f96fd434 service glance] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:23.163897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:23.164932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:23.165733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:23.166014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] PATH_INFO: `/v3/users/3f434638fb384c408fcf04690c368b48` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:23.167311 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f434638fb384c408fcf04690c368b48)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:23.169347 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:23.226264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052bdad-cd5e-428c-919b-066c67c44d4c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:23.227408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052bdad-cd5e-428c-919b-066c67c44d4c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:23.227906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052bdad-cd5e-428c-919b-066c67c44d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:23.234760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f434638fb384c408fcf04690c368b48'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:23.240957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f434638fb384c408fcf04690c368b48'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:23.241666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f434638fb384c408fcf04690c368b48'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:23.255245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2052bdad-cd5e-428c-919b-066c67c44d4c None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:23.256540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2052bdad-cd5e-428c-919b-066c67c44d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:23.256822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f434638fb384c408fcf04690c368b48'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:23.257572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-4be77b97-ff96-4e71-9abb-baa842acfd9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f434638fb384c408fcf04690c368b48'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:23.274848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1240/2472] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:23 2022] DELETE /identity/v3/users/3f434638fb384c408fcf04690c368b48 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:23.282333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beca64-e090-4fa9-99e0-8375dc0d9a02 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:23.282660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beca64-e090-4fa9-99e0-8375dc0d9a02 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:23.282911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beca64-e090-4fa9-99e0-8375dc0d9a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:23.310466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a8beca64-e090-4fa9-99e0-8375dc0d9a02 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:23.313728 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a8beca64-e090-4fa9-99e0-8375dc0d9a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:23.349538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1233/2473] 10.222.0.95 () {58 vars in 1009 bytes} [Wed Aug 31 19:26:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:23.371930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052bdad-cd5e-428c-919b-066c67c44d4c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:23.373817 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2052bdad-cd5e-428c-919b-066c67c44d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:23.394694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1241/2474] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:23.409611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8beca64-e090-4fa9-99e0-8375dc0d9a02 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:23.411556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a8beca64-e090-4fa9-99e0-8375dc0d9a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:23.423973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:23.441066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:23.484885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052bdad-cd5e-428c-919b-066c67c44d4c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:23.497117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8beca64-e090-4fa9-99e0-8375dc0d9a02 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:23.501022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:23.501703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:23.502052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:23.502401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:23.512236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:23.513735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:23.526531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1242/2475] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:23.558009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c1ae4-2204-46f9-9ae9-42351ac4e008 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:23.558630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1ae4-2204-46f9-9ae9-42351ac4e008 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:23.558916 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1ae4-2204-46f9-9ae9-42351ac4e008 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:23.559184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1ae4-2204-46f9-9ae9-42351ac4e008 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:23.630878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c1ae4-2204-46f9-9ae9-42351ac4e008 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:23.632335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c1ae4-2204-46f9-9ae9-42351ac4e008 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:23.641991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1234/2476] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:26:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:26:23.940912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:23.943489 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:23.976067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361bd7-683b-49c2-9f19-55ded26ece57 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:23.984999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:23.985742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:23.986114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:23.986286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] PATH_INFO: `/v3/projects/555935a811b447698269617fca8e7f6c` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:23.987725 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=555935a811b447698269617fca8e7f6c)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:23.990550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:24.099019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '555935a811b447698269617fca8e7f6c', 'request_id': 'req-7416e208-05e6-43e0-b197-56283a4febd3'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:24.113222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '555935a811b447698269617fca8e7f6c', 'request_id': 'req-7416e208-05e6-43e0-b197-56283a4febd3'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:24.132805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '555935a811b447698269617fca8e7f6c', 'request_id': 'req-7416e208-05e6-43e0-b197-56283a4febd3'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:24.159521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] The token cache is being invalidate because project 555935a811b447698269617fca8e7f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:24.159814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:24.160977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7416e208-05e6-43e0-b197-56283a4febd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:24.163098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1243/2477] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:23 2022] DELETE /identity/v3/projects/555935a811b447698269617fca8e7f6c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:24.169344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628af61-4ca0-490b-928b-5ac2243b3484 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.169969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628af61-4ca0-490b-928b-5ac2243b3484 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.170541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628af61-4ca0-490b-928b-5ac2243b3484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.203544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4628af61-4ca0-490b-928b-5ac2243b3484 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:24.204791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4628af61-4ca0-490b-928b-5ac2243b3484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:24.285135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1235/2478] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:24.301309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6957be-c19c-4170-a19e-bf08944a8e8a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.301582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6957be-c19c-4170-a19e-bf08944a8e8a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.301848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6957be-c19c-4170-a19e-bf08944a8e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.327771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-7e6957be-c19c-4170-a19e-bf08944a8e8a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:24.330179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7e6957be-c19c-4170-a19e-bf08944a8e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:24.341496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1244/2479] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:24.347502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628af61-4ca0-490b-928b-5ac2243b3484 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:24.359996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628af61-4ca0-490b-928b-5ac2243b3484 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:24.367844 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:24.368472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.368791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.369073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.370336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:24.370483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:24.372274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:24.385858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1236/2480] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:24.390730 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6957be-c19c-4170-a19e-bf08944a8e8a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:24.400291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6957be-c19c-4170-a19e-bf08944a8e8a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:24.405039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cf14f-c831-4b9a-a7f2-5bab86c46bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:24.406771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cf14f-c831-4b9a-a7f2-5bab86c46bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.407079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cf14f-c831-4b9a-a7f2-5bab86c46bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.407501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cf14f-c831-4b9a-a7f2-5bab86c46bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.408379 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cf14f-c831-4b9a-a7f2-5bab86c46bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:24.410122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cf14f-c831-4b9a-a7f2-5bab86c46bc6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:24.453832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1245/2481] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:24 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:24.457647 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:24.468311 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:24.479295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ecd80-9272-4be7-9d4d-bd5734ec3cba admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:24.624726 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1537f4-1547-45ae-88e1-0c1c3cad62ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:24.625351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1537f4-1547-45ae-88e1-0c1c3cad62ea admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.625633 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1537f4-1547-45ae-88e1-0c1c3cad62ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.626050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1537f4-1547-45ae-88e1-0c1c3cad62ea admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.626694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1537f4-1547-45ae-88e1-0c1c3cad62ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:24.634456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1537f4-1547-45ae-88e1-0c1c3cad62ea admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:24.662005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1237/2482] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:24 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:24.666282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cf14f-c831-4b9a-a7f2-5bab86c46bc6 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:24.685151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cf14f-c831-4b9a-a7f2-5bab86c46bc6 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:24.715289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aab73d-9e68-45a0-be25-292bec0e2d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:24.717179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aab73d-9e68-45a0-be25-292bec0e2d2e admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.718202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aab73d-9e68-45a0-be25-292bec0e2d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.725781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aab73d-9e68-45a0-be25-292bec0e2d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.725781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aab73d-9e68-45a0-be25-292bec0e2d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:24.726842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aab73d-9e68-45a0-be25-292bec0e2d2e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:24.756759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1246/2483] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:24.763854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1537f4-1547-45ae-88e1-0c1c3cad62ea admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:24.787917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1537f4-1547-45ae-88e1-0c1c3cad62ea admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:24.797919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ce48b-5846-44fa-aa50-9d2ac297ad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:24.799089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce48b-5846-44fa-aa50-9d2ac297ad87 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.800239 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce48b-5846-44fa-aa50-9d2ac297ad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.800853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ce48b-5846-44fa-aa50-9d2ac297ad87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.801909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ce48b-5846-44fa-aa50-9d2ac297ad87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:24.804888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ce48b-5846-44fa-aa50-9d2ac297ad87 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:24.823837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1238/2484] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:24.829679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aab73d-9e68-45a0-be25-292bec0e2d2e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:24.844796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aab73d-9e68-45a0-be25-292bec0e2d2e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:24.864106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c60f5d-8739-4eb5-8722-bacbb67f695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:24.864927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c60f5d-8739-4eb5-8722-bacbb67f695e admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.865622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c60f5d-8739-4eb5-8722-bacbb67f695e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.865900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c60f5d-8739-4eb5-8722-bacbb67f695e admin admin] PATH_INFO: `/v3/projects/c13fe37535534c59b7beb1f9862d4831/users/aec1fae9d0f94c18994e71411ba646ba/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.917123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c60f5d-8739-4eb5-8722-bacbb67f695e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13fe37535534c59b7beb1f9862d4831, user_id=aec1fae9d0f94c18994e71411ba646ba, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:24.920600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c60f5d-8739-4eb5-8722-bacbb67f695e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:24.939279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1247/2485] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:24 2022] PUT /identity/v3/projects/c13fe37535534c59b7beb1f9862d4831/users/aec1fae9d0f94c18994e71411ba646ba/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:24.951345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61ac6-2e87-4c5f-b7f9-8371e1766eb1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:24.951829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61ac6-2e87-4c5f-b7f9-8371e1766eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:24.952680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a61ac6-2e87-4c5f-b7f9-8371e1766eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:24.986233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e8a61ac6-2e87-4c5f-b7f9-8371e1766eb1 None None] MFA Rules not processed for user `aec1fae9d0f94c18994e71411ba646ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:24.987471 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e8a61ac6-2e87-4c5f-b7f9-8371e1766eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:25.065906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1239/2486] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:25.070976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c60f5d-8739-4eb5-8722-bacbb67f695e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:25.110606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:25.153767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c60f5d-8739-4eb5-8722-bacbb67f695e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:25.159729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f947f2-80a9-400d-92e9-42eebc55848d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:25.160346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f947f2-80a9-400d-92e9-42eebc55848d admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:25.160656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f947f2-80a9-400d-92e9-42eebc55848d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:25.160930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f947f2-80a9-400d-92e9-42eebc55848d admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:25.161573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f947f2-80a9-400d-92e9-42eebc55848d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:25.163510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f947f2-80a9-400d-92e9-42eebc55848d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:25.197318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1248/2487] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:25 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:25.201699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a61ac6-2e87-4c5f-b7f9-8371e1766eb1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:25.219130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a61ac6-2e87-4c5f-b7f9-8371e1766eb1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:25.298304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e3746-a4d9-459c-a3d8-f3311e41891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:25.299053 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e3746-a4d9-459c-a3d8-f3311e41891a admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:25.299248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e3746-a4d9-459c-a3d8-f3311e41891a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:25.299472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2e3746-a4d9-459c-a3d8-f3311e41891a admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:25.300192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e3746-a4d9-459c-a3d8-f3311e41891a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:25.302744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2e3746-a4d9-459c-a3d8-f3311e41891a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:25.325134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:25.332890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1240/2488] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:25 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:25.333839 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f947f2-80a9-400d-92e9-42eebc55848d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:25.346635 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f947f2-80a9-400d-92e9-42eebc55848d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:25.354640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264caca-a1b2-4b07-bc87-d58c182eb8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:25.355316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264caca-a1b2-4b07-bc87-d58c182eb8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:25.355626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264caca-a1b2-4b07-bc87-d58c182eb8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:25.355880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264caca-a1b2-4b07-bc87-d58c182eb8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:25.356638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264caca-a1b2-4b07-bc87-d58c182eb8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:25.358987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264caca-a1b2-4b07-bc87-d58c182eb8f5 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:25.370847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1249/2489] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:25.374775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e3746-a4d9-459c-a3d8-f3311e41891a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:25.385399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2e3746-a4d9-459c-a3d8-f3311e41891a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:25.396778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44835e61-d302-460b-97e0-0c637d191567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:25.397456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44835e61-d302-460b-97e0-0c637d191567 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:25.397743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44835e61-d302-460b-97e0-0c637d191567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:25.398083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44835e61-d302-460b-97e0-0c637d191567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:25.399127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44835e61-d302-460b-97e0-0c637d191567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:25.400748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44835e61-d302-460b-97e0-0c637d191567 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:25.413615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1241/2490] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:25.417768 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264caca-a1b2-4b07-bc87-d58c182eb8f5 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:25.434256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264caca-a1b2-4b07-bc87-d58c182eb8f5 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:25.446806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a425-7c6a-424b-9157-6488eb3f8b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:25.447897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a425-7c6a-424b-9157-6488eb3f8b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:25.448480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a425-7c6a-424b-9157-6488eb3f8b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:25.448999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9a425-7c6a-424b-9157-6488eb3f8b9f admin admin] PATH_INFO: `/v3/projects/18178d7e569e4b5b919d1cbce63e7fdf/users/2eb556e4ccb94170b346b050aab4580d/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:25.475442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9a425-7c6a-424b-9157-6488eb3f8b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18178d7e569e4b5b919d1cbce63e7fdf, user_id=2eb556e4ccb94170b346b050aab4580d, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:25.478282 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9a425-7c6a-424b-9157-6488eb3f8b9f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:25.495348 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1250/2491] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:25 2022] PUT /identity/v3/projects/18178d7e569e4b5b919d1cbce63e7fdf/users/2eb556e4ccb94170b346b050aab4580d/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:25.581683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b2038-9c4c-4758-bdb2-d3308207adbb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:25.581683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b2038-9c4c-4758-bdb2-d3308207adbb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:25.581683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b2038-9c4c-4758-bdb2-d3308207adbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:25.616924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-517b2038-9c4c-4758-bdb2-d3308207adbb None None] MFA Rules not processed for user `2eb556e4ccb94170b346b050aab4580d`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:25.618856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-517b2038-9c4c-4758-bdb2-d3308207adbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:25.700684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1242/2492] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:25.706270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a425-7c6a-424b-9157-6488eb3f8b9f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:25.812781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9a425-7c6a-424b-9157-6488eb3f8b9f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:25.824817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:25.825640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:25.825990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:25.826455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:25.828484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:25.837162 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:25.869647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1251/2493] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:25 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:25.873693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b2038-9c4c-4758-bdb2-d3308207adbb None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:25.885990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b2038-9c4c-4758-bdb2-d3308207adbb None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:26.027689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614899fa-2c86-4a18-951d-1a7968655e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:26.029460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614899fa-2c86-4a18-951d-1a7968655e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:26.031857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614899fa-2c86-4a18-951d-1a7968655e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:26.037079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614899fa-2c86-4a18-951d-1a7968655e5b admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:26.039307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614899fa-2c86-4a18-951d-1a7968655e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:26.042312 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614899fa-2c86-4a18-951d-1a7968655e5b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:26.059168 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:26.061944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:26.108381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1243/2494] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:25 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:26.112370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614899fa-2c86-4a18-951d-1a7968655e5b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:26.130386 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614899fa-2c86-4a18-951d-1a7968655e5b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:26.150333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9188b-c78b-404b-9e86-b7841ddb04f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:26.151917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9188b-c78b-404b-9e86-b7841ddb04f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:26.152744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9188b-c78b-404b-9e86-b7841ddb04f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:26.153428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9188b-c78b-404b-9e86-b7841ddb04f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:26.154580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9188b-c78b-404b-9e86-b7841ddb04f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:26.157429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9188b-c78b-404b-9e86-b7841ddb04f1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:26.158840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7249d-32e3-45b2-8b3b-2942762732b4 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:26.164414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:26.165400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:26.166586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:26.167063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:26.172152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:26.175136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1244/2495] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:26.179831 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9188b-c78b-404b-9e86-b7841ddb04f1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:26.193195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:26.194982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:26.195133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9188b-c78b-404b-9e86-b7841ddb04f1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:26.200635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e50268-96c5-4bb1-b86c-c546b0e47aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:26.201846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e50268-96c5-4bb1-b86c-c546b0e47aea admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:26.201846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e50268-96c5-4bb1-b86c-c546b0e47aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:26.202117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e50268-96c5-4bb1-b86c-c546b0e47aea admin admin] PATH_INFO: `/v3/projects/2e9e8bca0387498e8ecd98be545129e5/users/c5dc275b68cf4e328c8ceaed79a55574/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:26.211524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1252/2496] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:26.223485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e50268-96c5-4bb1-b86c-c546b0e47aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9e8bca0387498e8ecd98be545129e5, user_id=c5dc275b68cf4e328c8ceaed79a55574, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:26.226684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e50268-96c5-4bb1-b86c-c546b0e47aea admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:26.254266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1245/2497] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:26 2022] PUT /identity/v3/projects/2e9e8bca0387498e8ecd98be545129e5/users/c5dc275b68cf4e328c8ceaed79a55574/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:26.255615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:26.269432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cb5db-dac8-4adb-9bff-ca418d3faf9d service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:26.273832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:26.399840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a22b5-9b2c-4367-8852-18fc9a842f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:26.400475 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a22b5-9b2c-4367-8852-18fc9a842f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:26.400783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a22b5-9b2c-4367-8852-18fc9a842f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:26.401698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a22b5-9b2c-4367-8852-18fc9a842f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:26.402503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a22b5-9b2c-4367-8852-18fc9a842f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:26.404448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a22b5-9b2c-4367-8852-18fc9a842f8f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:26.423993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1253/2498] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:26.426090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e50268-96c5-4bb1-b86c-c546b0e47aea admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:26.439162 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e50268-96c5-4bb1-b86c-c546b0e47aea admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:26.450523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:26.453009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:26.453478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:26.454578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] PATH_INFO: `/v3/projects/2e9e8bca0387498e8ecd98be545129e5/users/c5dc275b68cf4e328c8ceaed79a55574/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:26.457074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9e8bca0387498e8ecd98be545129e5, user_id=c5dc275b68cf4e328c8ceaed79a55574, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:26.460324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:26.479701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1246/2499] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:26 2022] PUT /identity/v3/projects/2e9e8bca0387498e8ecd98be545129e5/users/c5dc275b68cf4e328c8ceaed79a55574/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:26.490518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4ee0a-7f23-4d61-bf3d-e058a3c436f4 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:26.490890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4ee0a-7f23-4d61-bf3d-e058a3c436f4 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:26.491058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4ee0a-7f23-4d61-bf3d-e058a3c436f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:26.540428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-4bc4ee0a-7f23-4d61-bf3d-e058a3c436f4 None None] MFA Rules not processed for user `c5dc275b68cf4e328c8ceaed79a55574`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:26.541840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4bc4ee0a-7f23-4d61-bf3d-e058a3c436f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:26.675935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1254/2500] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:26.688619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:26.690526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:26.733151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:26.849585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fcb41-aa34-41a9-8a9a-78c4fa90a1ff admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:26.860440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb112845-0303-4ac0-a237-1d36a8ae3aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:26.861205 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb112845-0303-4ac0-a237-1d36a8ae3aae service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:26.861561 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb112845-0303-4ac0-a237-1d36a8ae3aae service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:26.862287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb112845-0303-4ac0-a237-1d36a8ae3aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:26.973283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb112845-0303-4ac0-a237-1d36a8ae3aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:26.975749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb112845-0303-4ac0-a237-1d36a8ae3aae service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:26.987775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1247/2501] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:28.940807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4ee0a-7f23-4d61-bf3d-e058a3c436f4 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:28.943101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4bc4ee0a-7f23-4d61-bf3d-e058a3c436f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:28.971890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4ee0a-7f23-4d61-bf3d-e058a3c436f4 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:28.976094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:28.976834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:28.977118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:28.977398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:29.216044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:29.218192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:29.245299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1255/2502] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:30.397623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb112845-0303-4ac0-a237-1d36a8ae3aae service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:30.409284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb112845-0303-4ac0-a237-1d36a8ae3aae service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:30.415743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b9d1-d148-45f6-b6eb-b2314a59ab51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:30.420141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b9d1-d148-45f6-b6eb-b2314a59ab51 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:30.420141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b9d1-d148-45f6-b6eb-b2314a59ab51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:30.420141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b9d1-d148-45f6-b6eb-b2314a59ab51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:30.441095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0b9d1-d148-45f6-b6eb-b2314a59ab51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:30.442805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0b9d1-d148-45f6-b6eb-b2314a59ab51 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:30.462313 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1248/2503] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:31.904388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:31.908708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:31.996641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:32.086637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166548f-1b01-4e4b-a360-263cc3a7e6f1 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:32.093339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:32.094701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:32.095642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:32.096247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] PATH_INFO: `/v3/users/a288c513afd94adaa9f53f9f65dc9dd8` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:32.098431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a288c513afd94adaa9f53f9f65dc9dd8)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:32.103650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:32.249668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a288c513afd94adaa9f53f9f65dc9dd8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.270186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a288c513afd94adaa9f53f9f65dc9dd8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.271647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a288c513afd94adaa9f53f9f65dc9dd8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.345818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a288c513afd94adaa9f53f9f65dc9dd8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.363594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2f16e53-1e7f-45c3-bc31-5de2dbbe2835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a288c513afd94adaa9f53f9f65dc9dd8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.398683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1256/2504] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:31 2022] DELETE /identity/v3/users/a288c513afd94adaa9f53f9f65dc9dd8 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:32.709151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b9d1-d148-45f6-b6eb-b2314a59ab51 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:32.723114 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b9d1-d148-45f6-b6eb-b2314a59ab51 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:32.819211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:32.819839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:32.820142 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:32.820416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] PATH_INFO: `/v3/projects/6d1e607dfc7e4b089c7e744f4b1059e8` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:32.821608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d1e607dfc7e4b089c7e744f4b1059e8)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:32.823558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:32.870609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:32.874093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d1e607dfc7e4b089c7e744f4b1059e8', 'request_id': 'req-b19f321c-f8ae-4a56-97b7-f5c52221d396'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.882436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d1e607dfc7e4b089c7e744f4b1059e8', 'request_id': 'req-b19f321c-f8ae-4a56-97b7-f5c52221d396'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.889111 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d1e607dfc7e4b089c7e744f4b1059e8', 'request_id': 'req-b19f321c-f8ae-4a56-97b7-f5c52221d396'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.900441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] The token cache is being invalidate because project 6d1e607dfc7e4b089c7e744f4b1059e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:32.900768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.901516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19f321c-f8ae-4a56-97b7-f5c52221d396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:32.903268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1249/2505] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:32 2022] DELETE /identity/v3/projects/6d1e607dfc7e4b089c7e744f4b1059e8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:32.909037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aade81-8887-4907-9c1d-1eefc63cff78 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:32.909598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aade81-8887-4907-9c1d-1eefc63cff78 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:32.909598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aade81-8887-4907-9c1d-1eefc63cff78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:32.930481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-24aade81-8887-4907-9c1d-1eefc63cff78 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:32.931716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-24aade81-8887-4907-9c1d-1eefc63cff78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:32.999758 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1257/2506] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:33.008763 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2868c64-83f9-43fc-839d-92344bdb3f57 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.009052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2868c64-83f9-43fc-839d-92344bdb3f57 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.009330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2868c64-83f9-43fc-839d-92344bdb3f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.031828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c2868c64-83f9-43fc-839d-92344bdb3f57 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:33.033141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c2868c64-83f9-43fc-839d-92344bdb3f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:33.041457 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1250/2507] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:33.045803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aade81-8887-4907-9c1d-1eefc63cff78 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.056122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aade81-8887-4907-9c1d-1eefc63cff78 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.060912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.062082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.062361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.062638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.063291 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:33.063594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.065318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.078240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1258/2508] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:33.083048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2868c64-83f9-43fc-839d-92344bdb3f57 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.092298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2868c64-83f9-43fc-839d-92344bdb3f57 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.098387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db0674-2117-4de6-877a-787198f3740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.098985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db0674-2117-4de6-877a-787198f3740e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.099263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db0674-2117-4de6-877a-787198f3740e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.099530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db0674-2117-4de6-877a-787198f3740e admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.100155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db0674-2117-4de6-877a-787198f3740e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.102099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db0674-2117-4de6-877a-787198f3740e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.135184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1251/2509] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:33 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:33.140515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.150774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419271d-c2d3-411a-82a4-429d3b99eb9d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.184286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:33.215052 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29f4e1-5bd8-46d5-8828-5ac466db1816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.215675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29f4e1-5bd8-46d5-8828-5ac466db1816 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.215939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29f4e1-5bd8-46d5-8828-5ac466db1816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.216213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29f4e1-5bd8-46d5-8828-5ac466db1816 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.216871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29f4e1-5bd8-46d5-8828-5ac466db1816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.218936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29f4e1-5bd8-46d5-8828-5ac466db1816 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.240825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1259/2510] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:33 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:33.244900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db0674-2117-4de6-877a-787198f3740e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.256982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db0674-2117-4de6-877a-787198f3740e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.261889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e36fc-ae8a-409b-9c9a-516d45af6b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.262489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e36fc-ae8a-409b-9c9a-516d45af6b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.262766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e36fc-ae8a-409b-9c9a-516d45af6b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.263037 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e36fc-ae8a-409b-9c9a-516d45af6b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.263678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e36fc-ae8a-409b-9c9a-516d45af6b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.265483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e36fc-ae8a-409b-9c9a-516d45af6b21 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.277462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1252/2511] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:33.282564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29f4e1-5bd8-46d5-8828-5ac466db1816 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.292871 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29f4e1-5bd8-46d5-8828-5ac466db1816 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.298315 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f7bed-0c6b-42bc-a486-9e7042a99584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.299230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f7bed-0c6b-42bc-a486-9e7042a99584 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.299496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f7bed-0c6b-42bc-a486-9e7042a99584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.299817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f7bed-0c6b-42bc-a486-9e7042a99584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.300422 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f7bed-0c6b-42bc-a486-9e7042a99584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.302082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f7bed-0c6b-42bc-a486-9e7042a99584 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.313586 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1260/2512] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:33.318265 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e36fc-ae8a-409b-9c9a-516d45af6b21 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.333938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e36fc-ae8a-409b-9c9a-516d45af6b21 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.338529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02e61-9558-45ad-bb50-94416effb05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.339201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02e61-9558-45ad-bb50-94416effb05c admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.339476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02e61-9558-45ad-bb50-94416effb05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.339746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02e61-9558-45ad-bb50-94416effb05c admin admin] PATH_INFO: `/v3/projects/e12a6c0d4ecb4217a111a0859315727f/users/9ea027a923a946cd806e6775868e697f/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.355664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be02e61-9558-45ad-bb50-94416effb05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12a6c0d4ecb4217a111a0859315727f, user_id=9ea027a923a946cd806e6775868e697f, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.359330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be02e61-9558-45ad-bb50-94416effb05c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.379664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1253/2513] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:33 2022] PUT /identity/v3/projects/e12a6c0d4ecb4217a111a0859315727f/users/9ea027a923a946cd806e6775868e697f/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:33.390787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68049fb5-8c12-462a-a568-a536c2646d5a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.391008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68049fb5-8c12-462a-a568-a536c2646d5a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.391279 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68049fb5-8c12-462a-a568-a536c2646d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.422524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-68049fb5-8c12-462a-a568-a536c2646d5a None None] MFA Rules not processed for user `9ea027a923a946cd806e6775868e697f`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:33.423824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-68049fb5-8c12-462a-a568-a536c2646d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:33.517632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1261/2514] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:33.522333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02e61-9558-45ad-bb50-94416effb05c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.585582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02e61-9558-45ad-bb50-94416effb05c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.590564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f36ff-aea1-4aba-95c2-c40c4af37cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.591466 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f36ff-aea1-4aba-95c2-c40c4af37cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.592169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f36ff-aea1-4aba-95c2-c40c4af37cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.592169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f36ff-aea1-4aba-95c2-c40c4af37cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.593827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f36ff-aea1-4aba-95c2-c40c4af37cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.595394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f36ff-aea1-4aba-95c2-c40c4af37cd7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.628273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1254/2515] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:33 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:33.629605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68049fb5-8c12-462a-a568-a536c2646d5a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.644547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68049fb5-8c12-462a-a568-a536c2646d5a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.715796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796f009-5092-4cc1-8d2e-db9656b8d31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.716635 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796f009-5092-4cc1-8d2e-db9656b8d31e admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.717099 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796f009-5092-4cc1-8d2e-db9656b8d31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.717966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796f009-5092-4cc1-8d2e-db9656b8d31e admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.729118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796f009-5092-4cc1-8d2e-db9656b8d31e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.729118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796f009-5092-4cc1-8d2e-db9656b8d31e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.756818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1262/2516] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:33 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:33.758630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f36ff-aea1-4aba-95c2-c40c4af37cd7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.772814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f36ff-aea1-4aba-95c2-c40c4af37cd7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.780827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadbcfa-24b3-48f8-b278-885ccb94f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.781709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadbcfa-24b3-48f8-b278-885ccb94f1db admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.782012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadbcfa-24b3-48f8-b278-885ccb94f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.782287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadbcfa-24b3-48f8-b278-885ccb94f1db admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.782938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadbcfa-24b3-48f8-b278-885ccb94f1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.784549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadbcfa-24b3-48f8-b278-885ccb94f1db admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.796101 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1255/2517] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:33.801931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796f009-5092-4cc1-8d2e-db9656b8d31e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.812638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796f009-5092-4cc1-8d2e-db9656b8d31e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.817635 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c859b7b-c102-4570-8ba8-06028a0e2589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.818297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c859b7b-c102-4570-8ba8-06028a0e2589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.818627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c859b7b-c102-4570-8ba8-06028a0e2589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.818901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c859b7b-c102-4570-8ba8-06028a0e2589 admin admin] PATH_INFO: `/v3/projects/03350b0e63e4434db648896c7c8ec932/users/2d07f61d51564c0f9add7eb46beb9333/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.835415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c859b7b-c102-4570-8ba8-06028a0e2589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03350b0e63e4434db648896c7c8ec932, user_id=2d07f61d51564c0f9add7eb46beb9333, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.838115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c859b7b-c102-4570-8ba8-06028a0e2589 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.860817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1263/2518] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:33 2022] PUT /identity/v3/projects/03350b0e63e4434db648896c7c8ec932/users/2d07f61d51564c0f9add7eb46beb9333/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:33.860944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadbcfa-24b3-48f8-b278-885ccb94f1db admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.873228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadbcfa-24b3-48f8-b278-885ccb94f1db admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:33.933675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:33.962909 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f4326-66f3-4b87-9748-53c9ab5877ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:33.964720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f4326-66f3-4b87-9748-53c9ab5877ed admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:33.965046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f4326-66f3-4b87-9748-53c9ab5877ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:33.965329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f4326-66f3-4b87-9748-53c9ab5877ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:33.966002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f4326-66f3-4b87-9748-53c9ab5877ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:33.971141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f4326-66f3-4b87-9748-53c9ab5877ed admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:33.980612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1256/2519] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:33.984606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c859b7b-c102-4570-8ba8-06028a0e2589 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:33.999795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c859b7b-c102-4570-8ba8-06028a0e2589 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:34.006266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:34.007272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.007272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.007796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] PATH_INFO: `/v3/projects/03350b0e63e4434db648896c7c8ec932/users/2d07f61d51564c0f9add7eb46beb9333/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.009626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03350b0e63e4434db648896c7c8ec932, user_id=2d07f61d51564c0f9add7eb46beb9333, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:34.012317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:34.015870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:34.030457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1264/2520] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:33 2022] PUT /identity/v3/projects/03350b0e63e4434db648896c7c8ec932/users/2d07f61d51564c0f9add7eb46beb9333/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:34.037806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4a43-578a-42b9-8141-7219f534438a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.038136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4a43-578a-42b9-8141-7219f534438a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.038452 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4a43-578a-42b9-8141-7219f534438a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.069045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a54c4a43-578a-42b9-8141-7219f534438a None None] MFA Rules not processed for user `2d07f61d51564c0f9add7eb46beb9333`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:34.070595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a54c4a43-578a-42b9-8141-7219f534438a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:34.140626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1257/2521] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:34.151989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:34.153735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:34.217019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa2e4b-d6a8-48c3-9c26-a131caa38871 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:34.221405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565d5ab-60b5-4949-b5a4-3c9c5216ce4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:34.221967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d5ab-60b5-4949-b5a4-3c9c5216ce4d service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.222243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d5ab-60b5-4949-b5a4-3c9c5216ce4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.222516 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d5ab-60b5-4949-b5a4-3c9c5216ce4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.232146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d5ab-60b5-4949-b5a4-3c9c5216ce4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:34.234235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d5ab-60b5-4949-b5a4-3c9c5216ce4d service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:34.243855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1265/2522] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:34.314885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4a43-578a-42b9-8141-7219f534438a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:34.378227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4a43-578a-42b9-8141-7219f534438a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:34.382826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:34.383457 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.383733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.384039 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] PATH_INFO: `/v3/users/9ea027a923a946cd806e6775868e697f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.385166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea027a923a946cd806e6775868e697f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:34.387085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:34.439607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea027a923a946cd806e6775868e697f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.451415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea027a923a946cd806e6775868e697f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.452384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea027a923a946cd806e6775868e697f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.453065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea027a923a946cd806e6775868e697f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.454730 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:34.461698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea027a923a946cd806e6775868e697f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:34.475391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1258/2523] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:34 2022] DELETE /identity/v3/users/9ea027a923a946cd806e6775868e697f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:34.479868 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0aaef-ea3c-490c-89c7-813beb007430 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.480155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0aaef-ea3c-490c-89c7-813beb007430 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.480455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0aaef-ea3c-490c-89c7-813beb007430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.500534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e7f0aaef-ea3c-490c-89c7-813beb007430 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:34.502988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e7f0aaef-ea3c-490c-89c7-813beb007430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:34.574797 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1266/2524] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:34.585087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:34.586901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:34.653165 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78beca06-e255-40be-986f-2794303689f3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:34.657953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f94b3-8755-4696-bf9e-c7121c9a1dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:34.658568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f94b3-8755-4696-bf9e-c7121c9a1dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.658847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f94b3-8755-4696-bf9e-c7121c9a1dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.659194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f94b3-8755-4696-bf9e-c7121c9a1dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.669034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f94b3-8755-4696-bf9e-c7121c9a1dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:34.671008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f94b3-8755-4696-bf9e-c7121c9a1dc4 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:34.685094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1259/2525] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:34.958734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0aaef-ea3c-490c-89c7-813beb007430 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:34.960449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e7f0aaef-ea3c-490c-89c7-813beb007430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:34.978540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0aaef-ea3c-490c-89c7-813beb007430 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:34.984952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:34.985687 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.985913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.986184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] PATH_INFO: `/v3/projects/e12a6c0d4ecb4217a111a0859315727f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.987340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12a6c0d4ecb4217a111a0859315727f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:34.989286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:35.053804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12a6c0d4ecb4217a111a0859315727f', 'request_id': 'req-e058a1ce-5085-47a4-a0b6-6d05367e5241'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.060612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12a6c0d4ecb4217a111a0859315727f', 'request_id': 'req-e058a1ce-5085-47a4-a0b6-6d05367e5241'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.067121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12a6c0d4ecb4217a111a0859315727f', 'request_id': 'req-e058a1ce-5085-47a4-a0b6-6d05367e5241'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.089423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] The token cache is being invalidate because project e12a6c0d4ecb4217a111a0859315727f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:35.089714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.090360 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.091870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1267/2526] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:34 2022] DELETE /identity/v3/projects/e12a6c0d4ecb4217a111a0859315727f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:35.097415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f94b3-8755-4696-bf9e-c7121c9a1dc4 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:35.098502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:35.099865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:35.110766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f94b3-8755-4696-bf9e-c7121c9a1dc4 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:35.144619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:35.232367 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:35.235570 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.236223 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.236447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058a1ce-5085-47a4-a0b6-6d05367e5241 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:35.236725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] PATH_INFO: `/v3/users/2d07f61d51564c0f9add7eb46beb9333` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.238877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d07f61d51564c0f9add7eb46beb9333)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:35.243853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:35.244443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:35.244836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.245313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.245707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] PATH_INFO: `/v3/users/aec1fae9d0f94c18994e71411ba646ba` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.247571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec1fae9d0f94c18994e71411ba646ba)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:35.250683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:35.333089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec1fae9d0f94c18994e71411ba646ba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.346307 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec1fae9d0f94c18994e71411ba646ba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.347325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec1fae9d0f94c18994e71411ba646ba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.354262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d07f61d51564c0f9add7eb46beb9333'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.366635 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec1fae9d0f94c18994e71411ba646ba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.368417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d1f72361-028f-4452-a068-c5b31f4ae0e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec1fae9d0f94c18994e71411ba646ba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.380562 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d07f61d51564c0f9add7eb46beb9333'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.382354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d07f61d51564c0f9add7eb46beb9333'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.384678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d07f61d51564c0f9add7eb46beb9333'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.387333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1268/2527] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:35 2022] DELETE /identity/v3/users/aec1fae9d0f94c18994e71411ba646ba => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:35.393900 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8e299-1036-4f7e-915a-96db56a8d000 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.394222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8e299-1036-4f7e-915a-96db56a8d000 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.394536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8e299-1036-4f7e-915a-96db56a8d000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.401175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d07f61d51564c0f9add7eb46beb9333'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:35.423925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-44e8e299-1036-4f7e-915a-96db56a8d000 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:35.425256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-44e8e299-1036-4f7e-915a-96db56a8d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:35.435619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1260/2528] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:35 2022] DELETE /identity/v3/users/2d07f61d51564c0f9add7eb46beb9333 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:35.516986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1269/2529] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:35.532917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:35.534549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:35.541475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:35.652917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d66afe-c39a-431c-bfc5-275ed0d432d8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:35.659516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5bf45-ce60-4360-9aac-93be94079246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:35.660147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bf45-ce60-4360-9aac-93be94079246 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.660428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bf45-ce60-4360-9aac-93be94079246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.660791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bf45-ce60-4360-9aac-93be94079246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.767318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5bf45-ce60-4360-9aac-93be94079246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:35.769337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5bf45-ce60-4360-9aac-93be94079246 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:35.795877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1261/2530] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:35.892338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8e299-1036-4f7e-915a-96db56a8d000 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:35.894264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-44e8e299-1036-4f7e-915a-96db56a8d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:35.909562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8e299-1036-4f7e-915a-96db56a8d000 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:35.915660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:35.916762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:35.917831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:35.917831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] PATH_INFO: `/v3/projects/03350b0e63e4434db648896c7c8ec932` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:35.919370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03350b0e63e4434db648896c7c8ec932)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:35.922066 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.020342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:36.044417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03350b0e63e4434db648896c7c8ec932', 'request_id': 'req-8203200a-443c-483c-ab0b-8a3b445b23d9'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.056973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03350b0e63e4434db648896c7c8ec932', 'request_id': 'req-8203200a-443c-483c-ab0b-8a3b445b23d9'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.070559 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03350b0e63e4434db648896c7c8ec932', 'request_id': 'req-8203200a-443c-483c-ab0b-8a3b445b23d9'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.093827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] The token cache is being invalidate because project 03350b0e63e4434db648896c7c8ec932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:36.094209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.095134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8203200a-443c-483c-ab0b-8a3b445b23d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.105594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1270/2531] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:35 2022] DELETE /identity/v3/projects/03350b0e63e4434db648896c7c8ec932 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:36.112935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d3128-45cc-4315-bc54-9d38969a33e2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.113460 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d3128-45cc-4315-bc54-9d38969a33e2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.113568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d3128-45cc-4315-bc54-9d38969a33e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.147951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-901d3128-45cc-4315-bc54-9d38969a33e2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:36.149251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-901d3128-45cc-4315-bc54-9d38969a33e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.221213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1262/2532] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:36.229891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93a263-af4d-48bb-bea3-31f5c1a334fc None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.230173 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93a263-af4d-48bb-bea3-31f5c1a334fc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.230474 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93a263-af4d-48bb-bea3-31f5c1a334fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.252783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d3128-45cc-4315-bc54-9d38969a33e2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.254692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-901d3128-45cc-4315-bc54-9d38969a33e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.257558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-da93a263-af4d-48bb-bea3-31f5c1a334fc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:36.259272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-da93a263-af4d-48bb-bea3-31f5c1a334fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.275242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1271/2533] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:36.275696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d3128-45cc-4315-bc54-9d38969a33e2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.278341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93a263-af4d-48bb-bea3-31f5c1a334fc None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.288203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.289156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.289640 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.289981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] PATH_INFO: `/v3/projects/c13fe37535534c59b7beb1f9862d4831` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.291369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13fe37535534c59b7beb1f9862d4831)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.293400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.295935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93a263-af4d-48bb-bea3-31f5c1a334fc None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.302146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.302762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.303038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.303306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.303950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:36.304250 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.305904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.318857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1272/2534] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:26:36.323086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.338562 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228e3a4-1136-42fe-b266-7576c3c17fad admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.349106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13fe37535534c59b7beb1f9862d4831', 'request_id': 'req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.351635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:36.358294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13fe37535534c59b7beb1f9862d4831', 'request_id': 'req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.366877 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13fe37535534c59b7beb1f9862d4831', 'request_id': 'req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.381557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] The token cache is being invalidate because project c13fe37535534c59b7beb1f9862d4831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:36.381846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.382523 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.384619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1263/2535] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:36 2022] DELETE /identity/v3/projects/c13fe37535534c59b7beb1f9862d4831 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:36.388006 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.390157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.419328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.419964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.420419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.420539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.421969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.424505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.472287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1273/2536] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:36 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:36.476780 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.479392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.482335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea5405-4cbc-4f3d-8fa4-01e8892d2e75 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.552980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.553623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.553934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.554213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] PATH_INFO: `/v3/users/2eb556e4ccb94170b346b050aab4580d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.555356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb556e4ccb94170b346b050aab4580d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.557584 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.578633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f88a3-4bdb-4869-ab11-8b35100bec6f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.583541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bab5-9699-4be7-b0e7-7e7dd1709944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.584158 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bab5-9699-4be7-b0e7-7e7dd1709944 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.584444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bab5-9699-4be7-b0e7-7e7dd1709944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.584753 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bab5-9699-4be7-b0e7-7e7dd1709944 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.585400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83bab5-9699-4be7-b0e7-7e7dd1709944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.587569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83bab5-9699-4be7-b0e7-7e7dd1709944 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.609825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb556e4ccb94170b346b050aab4580d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.610707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1274/2537] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:36 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:36.614629 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bab5-9699-4be7-b0e7-7e7dd1709944 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.625666 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bab5-9699-4be7-b0e7-7e7dd1709944 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.626165 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb556e4ccb94170b346b050aab4580d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.628555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb556e4ccb94170b346b050aab4580d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.628555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb556e4ccb94170b346b050aab4580d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.631230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.631834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.632112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.632549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.633063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.634769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.642930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb556e4ccb94170b346b050aab4580d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:36.646488 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1275/2538] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:36.650426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.652197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.660472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1264/2539] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:36 2022] DELETE /identity/v3/users/2eb556e4ccb94170b346b050aab4580d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:36.665938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cff750-1af6-4eb6-bef4-4088230aff23 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.669855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:36.729470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206f9b1-8010-40c5-a0e1-4e1d08acddfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.730087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206f9b1-8010-40c5-a0e1-4e1d08acddfe admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.730371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206f9b1-8010-40c5-a0e1-4e1d08acddfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.730641 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206f9b1-8010-40c5-a0e1-4e1d08acddfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.731289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206f9b1-8010-40c5-a0e1-4e1d08acddfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.733438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206f9b1-8010-40c5-a0e1-4e1d08acddfe admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.744247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1276/2540] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:36.749160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.756255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:36.760668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951270f-8747-4b5d-884c-202b1af8bf47 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:36.767557 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e6d43-8e59-4e03-a9c0-5a7302814b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:36.768296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e6d43-8e59-4e03-a9c0-5a7302814b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.768885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e6d43-8e59-4e03-a9c0-5a7302814b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.769456 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e6d43-8e59-4e03-a9c0-5a7302814b6b admin admin] PATH_INFO: `/v3/projects/a3dd040eed3a4988afedbc12766d61cd/users/c131d424c8664665859c889a1ceb2bba/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.786853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e6d43-8e59-4e03-a9c0-5a7302814b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3dd040eed3a4988afedbc12766d61cd, user_id=c131d424c8664665859c889a1ceb2bba, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:36.789586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e6d43-8e59-4e03-a9c0-5a7302814b6b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:36.804904 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1265/2541] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:36 2022] PUT /identity/v3/projects/a3dd040eed3a4988afedbc12766d61cd/users/c131d424c8664665859c889a1ceb2bba/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:36.815841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb55b2-b753-447d-8f1e-b9cbf39985db None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:36.816405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb55b2-b753-447d-8f1e-b9cbf39985db None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:36.816712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb55b2-b753-447d-8f1e-b9cbf39985db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:36.850850 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b8bb55b2-b753-447d-8f1e-b9cbf39985db None None] MFA Rules not processed for user `c131d424c8664665859c889a1ceb2bba`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:36.852087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b8bb55b2-b753-447d-8f1e-b9cbf39985db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:36.921246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1277/2542] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:36.926389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e6d43-8e59-4e03-a9c0-5a7302814b6b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.956793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb55b2-b753-447d-8f1e-b9cbf39985db None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:36.958236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b8bb55b2-b753-447d-8f1e-b9cbf39985db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.020821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e6d43-8e59-4e03-a9c0-5a7302814b6b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.036920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaac2b1-649e-428b-a282-44194d87d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.037463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac2b1-649e-428b-a282-44194d87d259 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.037973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac2b1-649e-428b-a282-44194d87d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.038243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac2b1-649e-428b-a282-44194d87d259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.038891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaac2b1-649e-428b-a282-44194d87d259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.043233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaac2b1-649e-428b-a282-44194d87d259 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.057970 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb55b2-b753-447d-8f1e-b9cbf39985db None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.063762 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.064493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.064708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.064960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] PATH_INFO: `/v3/projects/18178d7e569e4b5b919d1cbce63e7fdf` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.066228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18178d7e569e4b5b919d1cbce63e7fdf)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.068556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.079074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1266/2543] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:36 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:37.083092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaac2b1-649e-428b-a282-44194d87d259 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.120832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:37.146927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18178d7e569e4b5b919d1cbce63e7fdf', 'request_id': 'req-db1a8442-9765-44f5-84a4-31a46ab9a7a2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.156429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18178d7e569e4b5b919d1cbce63e7fdf', 'request_id': 'req-db1a8442-9765-44f5-84a4-31a46ab9a7a2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.164171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18178d7e569e4b5b919d1cbce63e7fdf', 'request_id': 'req-db1a8442-9765-44f5-84a4-31a46ab9a7a2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.168897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaac2b1-649e-428b-a282-44194d87d259 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.185468 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] The token cache is being invalidate because project 18178d7e569e4b5b919d1cbce63e7fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:37.186411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.188013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.191211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1278/2544] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:36 2022] DELETE /identity/v3/projects/18178d7e569e4b5b919d1cbce63e7fdf => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:37.195592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.199769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.251105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.252032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.252665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.253105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.253936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.256314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.284660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1267/2545] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:37 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:37.285448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.287090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.294145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:37.302071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a8442-9765-44f5-84a4-31a46ab9a7a2 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.306878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7549a04-792d-4512-88ac-580c62ff318c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.307103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.307577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.308008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.308622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] PATH_INFO: `/v3/users/c5dc275b68cf4e328c8ceaed79a55574` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.309550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5dc275b68cf4e328c8ceaed79a55574)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.312070 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.315519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbe53a-2ac5-4ed0-bb0f-8d1ed17c2bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.316123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbe53a-2ac5-4ed0-bb0f-8d1ed17c2bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.316397 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbe53a-2ac5-4ed0-bb0f-8d1ed17c2bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.316860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbe53a-2ac5-4ed0-bb0f-8d1ed17c2bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.317509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbe53a-2ac5-4ed0-bb0f-8d1ed17c2bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.319422 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbe53a-2ac5-4ed0-bb0f-8d1ed17c2bd5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.332417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1268/2546] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:37.336251 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbe53a-2ac5-4ed0-bb0f-8d1ed17c2bd5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.346746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbe53a-2ac5-4ed0-bb0f-8d1ed17c2bd5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.351953 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.352718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.352978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.353269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] PATH_INFO: `/v3/projects/4c001a8a27204066b8df81b9db2980bd/users/6b55a63346f047abbd4c91922c0a169b/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.371049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c001a8a27204066b8df81b9db2980bd, user_id=6b55a63346f047abbd4c91922c0a169b, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.372585 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5dc275b68cf4e328c8ceaed79a55574'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.375150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.381867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5dc275b68cf4e328c8ceaed79a55574'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.382940 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5dc275b68cf4e328c8ceaed79a55574'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.392264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1269/2547] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:37 2022] PUT /identity/v3/projects/4c001a8a27204066b8df81b9db2980bd/users/6b55a63346f047abbd4c91922c0a169b/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:37.396093 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.398319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.404252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5dc275b68cf4e328c8ceaed79a55574'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.406369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5dc275b68cf4e328c8ceaed79a55574'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:37.425902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1279/2548] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:37 2022] DELETE /identity/v3/users/c5dc275b68cf4e328c8ceaed79a55574 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:37.496761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c117cb-3ea9-49c0-b972-b61c6a79f5fc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.577059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b740ecd-a4e7-4cbf-bbd2-4aaab7c4eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.577825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b740ecd-a4e7-4cbf-bbd2-4aaab7c4eb8e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.578108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b740ecd-a4e7-4cbf-bbd2-4aaab7c4eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.578385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b740ecd-a4e7-4cbf-bbd2-4aaab7c4eb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.579044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b740ecd-a4e7-4cbf-bbd2-4aaab7c4eb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.580717 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b740ecd-a4e7-4cbf-bbd2-4aaab7c4eb8e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.592616 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1270/2549] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:37.595704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.607395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e1053-da3c-4f0d-b138-a70ed0645236 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.613073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.613754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.614038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.614316 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] PATH_INFO: `/v3/projects/4c001a8a27204066b8df81b9db2980bd/users/6b55a63346f047abbd4c91922c0a169b/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.616953 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c001a8a27204066b8df81b9db2980bd, user_id=6b55a63346f047abbd4c91922c0a169b, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.619591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.631464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1280/2550] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:37 2022] PUT /identity/v3/projects/4c001a8a27204066b8df81b9db2980bd/users/6b55a63346f047abbd4c91922c0a169b/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:37.643733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa833b-90a6-4ddf-bc2a-ac65eff859e3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.643733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa833b-90a6-4ddf-bc2a-ac65eff859e3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.644050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa833b-90a6-4ddf-bc2a-ac65eff859e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.674755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d8fa833b-90a6-4ddf-bc2a-ac65eff859e3 None None] MFA Rules not processed for user `6b55a63346f047abbd4c91922c0a169b`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:37.675976 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d8fa833b-90a6-4ddf-bc2a-ac65eff859e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.749933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.755426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.778622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:37.795329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1271/2551] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:37.806827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa833b-90a6-4ddf-bc2a-ac65eff859e3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.808382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d8fa833b-90a6-4ddf-bc2a-ac65eff859e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.808590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:37.867405 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bea82a-17fd-415b-9654-bb35a9263dc7 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.886975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.886975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.886975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.886975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] PATH_INFO: `/v3/projects/2e9e8bca0387498e8ecd98be545129e5` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.887614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e9e8bca0387498e8ecd98be545129e5)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.908710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa833b-90a6-4ddf-bc2a-ac65eff859e3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.908938 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.922569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d135912-4686-4ad7-b8cf-950d71cbd2fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.925638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d135912-4686-4ad7-b8cf-950d71cbd2fa service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.927034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d135912-4686-4ad7-b8cf-950d71cbd2fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.927652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d135912-4686-4ad7-b8cf-950d71cbd2fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.932731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0d135912-4686-4ad7-b8cf-950d71cbd2fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.046657 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e9e8bca0387498e8ecd98be545129e5', 'request_id': 'req-85a6c690-49d1-4d47-a538-98cab8279219'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.054124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e9e8bca0387498e8ecd98be545129e5', 'request_id': 'req-85a6c690-49d1-4d47-a538-98cab8279219'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.056624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d135912-4686-4ad7-b8cf-950d71cbd2fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.058385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d135912-4686-4ad7-b8cf-950d71cbd2fa service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.061249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e9e8bca0387498e8ecd98be545129e5', 'request_id': 'req-85a6c690-49d1-4d47-a538-98cab8279219'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.070582 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1272/2552] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:38.071943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2994d69-5699-49aa-b82c-2187daa6ac62 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.072385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2994d69-5699-49aa-b82c-2187daa6ac62 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.072748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2994d69-5699-49aa-b82c-2187daa6ac62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.082001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] The token cache is being invalidate because project 2e9e8bca0387498e8ecd98be545129e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:38.082308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.084098 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-85a6c690-49d1-4d47-a538-98cab8279219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:38.086003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1281/2553] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:37 2022] DELETE /identity/v3/projects/2e9e8bca0387498e8ecd98be545129e5 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:38.091848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10c303-8907-4839-a6e0-e91e895fb1e6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.093041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10c303-8907-4839-a6e0-e91e895fb1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.093041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10c303-8907-4839-a6e0-e91e895fb1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.101813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c2994d69-5699-49aa-b82c-2187daa6ac62 None None] MFA Rules not processed for user `bc1911f5d10c4fe38b6a6c4a0950726a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:38.103497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c2994d69-5699-49aa-b82c-2187daa6ac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.119507 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fe10c303-8907-4839-a6e0-e91e895fb1e6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:38.120955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fe10c303-8907-4839-a6e0-e91e895fb1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.171656 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1273/2554] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:38.195971 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1282/2555] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:38.204331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e18d3-a9ad-4a68-ab6a-43cef57818ff None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.204646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e18d3-a9ad-4a68-ab6a-43cef57818ff None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.204927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e18d3-a9ad-4a68-ab6a-43cef57818ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.228863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-369e18d3-a9ad-4a68-ab6a-43cef57818ff None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:38.230089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-369e18d3-a9ad-4a68-ab6a-43cef57818ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.241653 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1274/2556] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:38.247399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10c303-8907-4839-a6e0-e91e895fb1e6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.269047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10c303-8907-4839-a6e0-e91e895fb1e6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.277701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.278326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.279410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.279410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.280384 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:38.281057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.284707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.303427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1283/2557] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:38.307625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e18d3-a9ad-4a68-ab6a-43cef57818ff None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.318482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e18d3-a9ad-4a68-ab6a-43cef57818ff None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.324768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ab96-8e89-4327-bb47-ec197df5612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.325293 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ab96-8e89-4327-bb47-ec197df5612d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.325735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ab96-8e89-4327-bb47-ec197df5612d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.326164 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ab96-8e89-4327-bb47-ec197df5612d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.327485 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060ab96-8e89-4327-bb47-ec197df5612d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.330217 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060ab96-8e89-4327-bb47-ec197df5612d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.366830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1275/2558] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:38.371219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.383579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91fade-ada6-4fb9-b696-809f14fb36fb admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.442390 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:38.470783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668fd4f-780c-421b-b049-ac7ff43dcc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.471679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668fd4f-780c-421b-b049-ac7ff43dcc09 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.471919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668fd4f-780c-421b-b049-ac7ff43dcc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.472194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668fd4f-780c-421b-b049-ac7ff43dcc09 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.472886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668fd4f-780c-421b-b049-ac7ff43dcc09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.475138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668fd4f-780c-421b-b049-ac7ff43dcc09 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.508784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1284/2559] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:38.513185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ab96-8e89-4327-bb47-ec197df5612d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.526447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ab96-8e89-4327-bb47-ec197df5612d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.532941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dafbc-631b-47f2-bfef-356aa40f58df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.533533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dafbc-631b-47f2-bfef-356aa40f58df admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.533888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dafbc-631b-47f2-bfef-356aa40f58df admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.534074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dafbc-631b-47f2-bfef-356aa40f58df admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.534840 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dafbc-631b-47f2-bfef-356aa40f58df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.537179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dafbc-631b-47f2-bfef-356aa40f58df admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.549768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1276/2560] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:38.553991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668fd4f-780c-421b-b049-ac7ff43dcc09 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.569187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668fd4f-780c-421b-b049-ac7ff43dcc09 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.593521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a238311-a2d6-4857-94d8-03cd8fcedac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.594273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a238311-a2d6-4857-94d8-03cd8fcedac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.594620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a238311-a2d6-4857-94d8-03cd8fcedac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.594956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a238311-a2d6-4857-94d8-03cd8fcedac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.595695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a238311-a2d6-4857-94d8-03cd8fcedac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.599522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a238311-a2d6-4857-94d8-03cd8fcedac8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.612620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1285/2561] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:38.616603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dafbc-631b-47f2-bfef-356aa40f58df admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.629655 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dafbc-631b-47f2-bfef-356aa40f58df admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.634887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac573f-324e-418e-a63e-579aee9bd15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.635620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac573f-324e-418e-a63e-579aee9bd15b admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.635903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac573f-324e-418e-a63e-579aee9bd15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.636175 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac573f-324e-418e-a63e-579aee9bd15b admin admin] PATH_INFO: `/v3/projects/b22a23bf00124e02962153b5cccffc59/users/3b36cbbe9e184ebeab62e8f3025c9b22/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.650359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:38.653900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac573f-324e-418e-a63e-579aee9bd15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b22a23bf00124e02962153b5cccffc59, user_id=3b36cbbe9e184ebeab62e8f3025c9b22, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.658060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac573f-324e-418e-a63e-579aee9bd15b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.674389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1277/2562] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:38 2022] PUT /identity/v3/projects/b22a23bf00124e02962153b5cccffc59/users/3b36cbbe9e184ebeab62e8f3025c9b22/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:38.686855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4a728-4adf-4f4a-ac2c-6d485cdcb11c None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.687178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4a728-4adf-4f4a-ac2c-6d485cdcb11c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.687457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4a728-4adf-4f4a-ac2c-6d485cdcb11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.722716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-96d4a728-4adf-4f4a-ac2c-6d485cdcb11c None None] MFA Rules not processed for user `3b36cbbe9e184ebeab62e8f3025c9b22`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:38.723978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-96d4a728-4adf-4f4a-ac2c-6d485cdcb11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:38.803548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1286/2563] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:38.804803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac573f-324e-418e-a63e-579aee9bd15b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.897303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac573f-324e-418e-a63e-579aee9bd15b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:38.902281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc224dc9-b520-4123-89a8-68102ac13dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:38.902939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc224dc9-b520-4123-89a8-68102ac13dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:38.903362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc224dc9-b520-4123-89a8-68102ac13dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:38.903638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc224dc9-b520-4123-89a8-68102ac13dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:38.904468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc224dc9-b520-4123-89a8-68102ac13dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:38.906756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc224dc9-b520-4123-89a8-68102ac13dc2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:38.944912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1278/2564] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:38.948919 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4a728-4adf-4f4a-ac2c-6d485cdcb11c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:38.962271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4a728-4adf-4f4a-ac2c-6d485cdcb11c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.034606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027c51-59e8-4f7a-91c8-b8758b8988aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.035229 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027c51-59e8-4f7a-91c8-b8758b8988aa admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.035565 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027c51-59e8-4f7a-91c8-b8758b8988aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.035867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027c51-59e8-4f7a-91c8-b8758b8988aa admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.036729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd027c51-59e8-4f7a-91c8-b8758b8988aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.040476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd027c51-59e8-4f7a-91c8-b8758b8988aa admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.076612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1287/2565] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:38 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:39.078550 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc224dc9-b520-4123-89a8-68102ac13dc2 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.088971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc224dc9-b520-4123-89a8-68102ac13dc2 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.094540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480ba4-d430-462d-a7b0-274c4eeda66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.095183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480ba4-d430-462d-a7b0-274c4eeda66f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.095659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480ba4-d430-462d-a7b0-274c4eeda66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.096003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480ba4-d430-462d-a7b0-274c4eeda66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.096614 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480ba4-d430-462d-a7b0-274c4eeda66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.098309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480ba4-d430-462d-a7b0-274c4eeda66f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.112085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1279/2566] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:39.115658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027c51-59e8-4f7a-91c8-b8758b8988aa admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.126898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027c51-59e8-4f7a-91c8-b8758b8988aa admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.133017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33f825-279b-4406-a2a4-dd63e7a0770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.133704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f825-279b-4406-a2a4-dd63e7a0770b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.134054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f825-279b-4406-a2a4-dd63e7a0770b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.134393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f825-279b-4406-a2a4-dd63e7a0770b admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.135109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33f825-279b-4406-a2a4-dd63e7a0770b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.136841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33f825-279b-4406-a2a4-dd63e7a0770b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.148335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1288/2567] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:39.152898 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480ba4-d430-462d-a7b0-274c4eeda66f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.164197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480ba4-d430-462d-a7b0-274c4eeda66f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.169063 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439f1f7-3120-4788-9f78-a567633baae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.169810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f1f7-3120-4788-9f78-a567633baae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.170161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f1f7-3120-4788-9f78-a567633baae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.171196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439f1f7-3120-4788-9f78-a567633baae8 admin admin] PATH_INFO: `/v3/projects/1e3b1ba52a3d42b7983a27288564275c/users/c7abf4b54031409bb7bcc4787dd38e4e/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.196547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439f1f7-3120-4788-9f78-a567633baae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3b1ba52a3d42b7983a27288564275c, user_id=c7abf4b54031409bb7bcc4787dd38e4e, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.200001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439f1f7-3120-4788-9f78-a567633baae8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.218870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1280/2568] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:39 2022] PUT /identity/v3/projects/1e3b1ba52a3d42b7983a27288564275c/users/c7abf4b54031409bb7bcc4787dd38e4e/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:39.230615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615c063-81bd-43b5-aef4-4dd38b6146bb None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.230615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615c063-81bd-43b5-aef4-4dd38b6146bb None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.230975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615c063-81bd-43b5-aef4-4dd38b6146bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.268064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8615c063-81bd-43b5-aef4-4dd38b6146bb None None] MFA Rules not processed for user `c7abf4b54031409bb7bcc4787dd38e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:39.269345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8615c063-81bd-43b5-aef4-4dd38b6146bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:39.297726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:39.356285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1289/2569] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:39.361187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439f1f7-3120-4788-9f78-a567633baae8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.450756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439f1f7-3120-4788-9f78-a567633baae8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.457860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a025b4-27a1-4b3d-ab8f-ab7a2271b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.457860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a025b4-27a1-4b3d-ab8f-ab7a2271b743 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.458104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a025b4-27a1-4b3d-ab8f-ab7a2271b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.458431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a025b4-27a1-4b3d-ab8f-ab7a2271b743 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.459086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a025b4-27a1-4b3d-ab8f-ab7a2271b743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.461189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a025b4-27a1-4b3d-ab8f-ab7a2271b743 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.491484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1281/2570] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:39.495722 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615c063-81bd-43b5-aef4-4dd38b6146bb None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.508177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615c063-81bd-43b5-aef4-4dd38b6146bb None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.609154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233967d-03fa-454c-a490-571e6cf56f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.610011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233967d-03fa-454c-a490-571e6cf56f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.610220 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233967d-03fa-454c-a490-571e6cf56f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.610571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233967d-03fa-454c-a490-571e6cf56f17 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.611602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233967d-03fa-454c-a490-571e6cf56f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.614552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233967d-03fa-454c-a490-571e6cf56f17 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.649697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1290/2571] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:39.651446 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a025b4-27a1-4b3d-ab8f-ab7a2271b743 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.663625 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a025b4-27a1-4b3d-ab8f-ab7a2271b743 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.668787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b925e-e988-4507-a195-06ba9600985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.669396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b925e-e988-4507-a195-06ba9600985a admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.669678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b925e-e988-4507-a195-06ba9600985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.669947 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b925e-e988-4507-a195-06ba9600985a admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.670690 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b925e-e988-4507-a195-06ba9600985a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.672295 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b925e-e988-4507-a195-06ba9600985a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.684701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1282/2572] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:39.688689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233967d-03fa-454c-a490-571e6cf56f17 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.699655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233967d-03fa-454c-a490-571e6cf56f17 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.707416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db117df2-73a4-47b6-b60c-941c88f61caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.708322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db117df2-73a4-47b6-b60c-941c88f61caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.708640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db117df2-73a4-47b6-b60c-941c88f61caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.708921 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db117df2-73a4-47b6-b60c-941c88f61caa admin admin] PATH_INFO: `/v3/projects/5ab5df60b248465e81a9e5b119040417/users/1c6f5916087f46afb344f4446af13d28/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.725752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db117df2-73a4-47b6-b60c-941c88f61caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab5df60b248465e81a9e5b119040417, user_id=1c6f5916087f46afb344f4446af13d28, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.739646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db117df2-73a4-47b6-b60c-941c88f61caa admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.761566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1291/2573] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:39 2022] PUT /identity/v3/projects/5ab5df60b248465e81a9e5b119040417/users/1c6f5916087f46afb344f4446af13d28/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:39.766288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b925e-e988-4507-a195-06ba9600985a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.786806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b925e-e988-4507-a195-06ba9600985a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.799974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:39.851381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fee076-6b51-4aaf-9525-ecf5f7dcfc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.851986 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fee076-6b51-4aaf-9525-ecf5f7dcfc72 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.852304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fee076-6b51-4aaf-9525-ecf5f7dcfc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.852619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fee076-6b51-4aaf-9525-ecf5f7dcfc72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.853339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fee076-6b51-4aaf-9525-ecf5f7dcfc72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.854919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fee076-6b51-4aaf-9525-ecf5f7dcfc72 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.868097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1283/2574] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:39.872267 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db117df2-73a4-47b6-b60c-941c88f61caa admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:39.892123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db117df2-73a4-47b6-b60c-941c88f61caa admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:39.897908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:39.898651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.899008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.899417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] PATH_INFO: `/v3/projects/5ab5df60b248465e81a9e5b119040417/users/1c6f5916087f46afb344f4446af13d28/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.901658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab5df60b248465e81a9e5b119040417, user_id=1c6f5916087f46afb344f4446af13d28, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:39.904437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:39.917157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1292/2575] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:39 2022] PUT /identity/v3/projects/5ab5df60b248465e81a9e5b119040417/users/1c6f5916087f46afb344f4446af13d28/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:39.929222 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25350ca8-ddac-49f6-b5bf-5f979d452627 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:39.929484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25350ca8-ddac-49f6-b5bf-5f979d452627 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:39.929830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25350ca8-ddac-49f6-b5bf-5f979d452627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:39.959158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-25350ca8-ddac-49f6-b5bf-5f979d452627 None None] MFA Rules not processed for user `1c6f5916087f46afb344f4446af13d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:39.960415 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-25350ca8-ddac-49f6-b5bf-5f979d452627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:40.031547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1284/2576] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:40.043502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:40.045223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:40.106365 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d2ad4-706a-456f-a0ce-49ebd387fa49 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:40.111033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017648d-31d8-4844-869f-86a5bf817b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:40.111646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017648d-31d8-4844-869f-86a5bf817b4e service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:40.111926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017648d-31d8-4844-869f-86a5bf817b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:40.112194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017648d-31d8-4844-869f-86a5bf817b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:40.148866 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:40.182498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017648d-31d8-4844-869f-86a5bf817b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:40.183127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017648d-31d8-4844-869f-86a5bf817b4e service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:40.191989 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1293/2577] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:41.741514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25350ca8-ddac-49f6-b5bf-5f979d452627 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:41.742984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-25350ca8-ddac-49f6-b5bf-5f979d452627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:41.757765 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25350ca8-ddac-49f6-b5bf-5f979d452627 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:41.762854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c4e3-e83a-40e9-a44e-defb744b3291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:41.763479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c4e3-e83a-40e9-a44e-defb744b3291 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:41.763761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c4e3-e83a-40e9-a44e-defb744b3291 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:41.764041 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c4e3-e83a-40e9-a44e-defb744b3291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:41.840896 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c4e3-e83a-40e9-a44e-defb744b3291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:41.842437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c4e3-e83a-40e9-a44e-defb744b3291 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:41.854672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1285/2578] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:42.754622 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017648d-31d8-4844-869f-86a5bf817b4e service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.765781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017648d-31d8-4844-869f-86a5bf817b4e service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.770188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.770801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.771083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.771359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.773201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:42.806819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c4e3-e83a-40e9-a44e-defb744b3291 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:42.816551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c4e3-e83a-40e9-a44e-defb744b3291 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:42.820638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daccdd2-8482-4c77-a4dc-f761cee7fbbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:42.821253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daccdd2-8482-4c77-a4dc-f761cee7fbbc service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.821528 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daccdd2-8482-4c77-a4dc-f761cee7fbbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.821806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daccdd2-8482-4c77-a4dc-f761cee7fbbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.832316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daccdd2-8482-4c77-a4dc-f761cee7fbbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.833842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daccdd2-8482-4c77-a4dc-f761cee7fbbc service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.844878 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1286/2579] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:42.863283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:42.864859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:42.892522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483853d-6bb2-4426-ad46-b5f7b1972631 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:42.893212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483853d-6bb2-4426-ad46-b5f7b1972631 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:42.893728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483853d-6bb2-4426-ad46-b5f7b1972631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:42.903419 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1294/2580] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:42.925153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d483853d-6bb2-4426-ad46-b5f7b1972631 None None] MFA Rules not processed for user `8a1dcd805548450cb3411c62083a4bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:42.927394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d483853d-6bb2-4426-ad46-b5f7b1972631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:42.979589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:43.048210 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1287/2581] 10.222.0.95 () {58 vars in 967 bytes} [Wed Aug 31 19:26:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:43.077010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:43.080112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:43.179011 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effd22d-e01c-4d27-bd7a-588b0b85fc00 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:43.185204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6498806-af53-4e58-a2eb-b408fd0b1e68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac9441fcf60412dbc33e0589a86c8ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:43.185828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6498806-af53-4e58-a2eb-b408fd0b1e68 service swift] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:43.186107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6498806-af53-4e58-a2eb-b408fd0b1e68 service swift] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:43.186430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6498806-af53-4e58-a2eb-b408fd0b1e68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:43.197899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6498806-af53-4e58-a2eb-b408fd0b1e68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:43.199711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6498806-af53-4e58-a2eb-b408fd0b1e68 service swift] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:43.211046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1295/2582] 10.222.0.95 () {60 vars in 1384 bytes} [Wed Aug 31 19:26:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:26:47.264892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483853d-6bb2-4426-ad46-b5f7b1972631 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:47.339700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483853d-6bb2-4426-ad46-b5f7b1972631 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:47.344751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:47.345351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.345830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.346107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] PATH_INFO: `/v3/users/3b36cbbe9e184ebeab62e8f3025c9b22` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.347373 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b36cbbe9e184ebeab62e8f3025c9b22)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:47.349391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:47.412938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b36cbbe9e184ebeab62e8f3025c9b22'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.431178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b36cbbe9e184ebeab62e8f3025c9b22'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.431901 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b36cbbe9e184ebeab62e8f3025c9b22'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.432517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b36cbbe9e184ebeab62e8f3025c9b22'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.441473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b36cbbe9e184ebeab62e8f3025c9b22'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:47.455396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1288/2583] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:47 2022] DELETE /identity/v3/users/3b36cbbe9e184ebeab62e8f3025c9b22 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:47.460157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10daa-a640-4929-9bc3-446a73536675 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.460785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10daa-a640-4929-9bc3-446a73536675 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.460785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10daa-a640-4929-9bc3-446a73536675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.481308 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-41c10daa-a640-4929-9bc3-446a73536675 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:47.482513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-41c10daa-a640-4929-9bc3-446a73536675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:47.532033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:47.545672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1296/2584] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:47.556374 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:47.557905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:47.614177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:47.620750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d5242-1656-4eb9-aa1f-8b91c49d47c7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:47.628112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064317-9be4-4b67-9efb-efee7b55358a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:47.628783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064317-9be4-4b67-9efb-efee7b55358a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.629067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064317-9be4-4b67-9efb-efee7b55358a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.629341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064317-9be4-4b67-9efb-efee7b55358a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.639326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3064317-9be4-4b67-9efb-efee7b55358a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:47.641524 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3064317-9be4-4b67-9efb-efee7b55358a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:47.651612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1289/2585] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:47.925156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c10daa-a640-4929-9bc3-446a73536675 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:47.926647 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-41c10daa-a640-4929-9bc3-446a73536675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:47.940619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c10daa-a640-4929-9bc3-446a73536675 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:47.945642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:47.946416 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:47.946763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:47.947090 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] PATH_INFO: `/v3/projects/b22a23bf00124e02962153b5cccffc59` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:47.948336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b22a23bf00124e02962153b5cccffc59)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:47.950614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.001368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b22a23bf00124e02962153b5cccffc59', 'request_id': 'req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.008484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b22a23bf00124e02962153b5cccffc59', 'request_id': 'req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.014086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b22a23bf00124e02962153b5cccffc59', 'request_id': 'req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.026476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] The token cache is being invalidate because project b22a23bf00124e02962153b5cccffc59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:48.026766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.027382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.028929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1297/2586] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:47 2022] DELETE /identity/v3/projects/b22a23bf00124e02962153b5cccffc59 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:48.033383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064317-9be4-4b67-9efb-efee7b55358a service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.043480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064317-9be4-4b67-9efb-efee7b55358a service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.114105 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.114958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.115315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.115507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] PATH_INFO: `/v3/users/c7abf4b54031409bb7bcc4787dd38e4e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.116723 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7abf4b54031409bb7bcc4787dd38e4e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.118722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.151433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.153132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.167245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284bc8-a54c-4cc8-bc7d-6e135d2edf80 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.173494 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.174146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.174471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.174936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] PATH_INFO: `/v3/users/c131d424c8664665859c889a1ceb2bba` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.177461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c131d424c8664665859c889a1ceb2bba)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.178156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.187540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7abf4b54031409bb7bcc4787dd38e4e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.203022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7abf4b54031409bb7bcc4787dd38e4e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.203931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7abf4b54031409bb7bcc4787dd38e4e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.204884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7abf4b54031409bb7bcc4787dd38e4e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.214735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-576dd190-af33-49c4-afe2-f6cf728d70c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7abf4b54031409bb7bcc4787dd38e4e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.235263 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1290/2587] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:48 2022] DELETE /identity/v3/users/c7abf4b54031409bb7bcc4787dd38e4e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:48.258185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c131d424c8664665859c889a1ceb2bba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.266691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c131d424c8664665859c889a1ceb2bba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.267699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c131d424c8664665859c889a1ceb2bba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.284087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c131d424c8664665859c889a1ceb2bba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.284922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c131d424c8664665859c889a1ceb2bba'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.307877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1298/2588] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:48 2022] DELETE /identity/v3/users/c131d424c8664665859c889a1ceb2bba => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:48.313721 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555a4d4-5f4f-4f73-a730-89bc15ba3d50 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.314547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555a4d4-5f4f-4f73-a730-89bc15ba3d50 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.315114 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555a4d4-5f4f-4f73-a730-89bc15ba3d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.340179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1555a4d4-5f4f-4f73-a730-89bc15ba3d50 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:48.341669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1555a4d4-5f4f-4f73-a730-89bc15ba3d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.413588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1291/2589] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:48.423782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.425248 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.488627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:48.496195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b384a6-8427-4f1c-8866-1813b7b7252c admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.502094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa872e-272b-419d-9ee2-69bae5b8f4d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.503203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa872e-272b-419d-9ee2-69bae5b8f4d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.503203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa872e-272b-419d-9ee2-69bae5b8f4d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.503376 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa872e-272b-419d-9ee2-69bae5b8f4d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.517750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa872e-272b-419d-9ee2-69bae5b8f4d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.519501 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa872e-272b-419d-9ee2-69bae5b8f4d6 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.533650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1299/2590] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:48.559951 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555a4d4-5f4f-4f73-a730-89bc15ba3d50 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.562054 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1555a4d4-5f4f-4f73-a730-89bc15ba3d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.576106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555a4d4-5f4f-4f73-a730-89bc15ba3d50 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.582292 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.582971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.583529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.583529 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] PATH_INFO: `/v3/projects/1e3b1ba52a3d42b7983a27288564275c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.584790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3b1ba52a3d42b7983a27288564275c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.587007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.589905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:48.652691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3b1ba52a3d42b7983a27288564275c', 'request_id': 'req-4666463d-758b-41fc-8f11-f42f8005e3e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.664274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3b1ba52a3d42b7983a27288564275c', 'request_id': 'req-4666463d-758b-41fc-8f11-f42f8005e3e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.672489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3b1ba52a3d42b7983a27288564275c', 'request_id': 'req-4666463d-758b-41fc-8f11-f42f8005e3e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.683199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] The token cache is being invalidate because project 1e3b1ba52a3d42b7983a27288564275c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:48.683488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.684171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.685965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1292/2591] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:48 2022] DELETE /identity/v3/projects/1e3b1ba52a3d42b7983a27288564275c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:48.691683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa872e-272b-419d-9ee2-69bae5b8f4d6 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.704272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa872e-272b-419d-9ee2-69bae5b8f4d6 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.774406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.775122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.775401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.775676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] PATH_INFO: `/v3/users/1c6f5916087f46afb344f4446af13d28` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.777140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6f5916087f46afb344f4446af13d28)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.779075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:48.832352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6f5916087f46afb344f4446af13d28'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.839688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6f5916087f46afb344f4446af13d28'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.840451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6f5916087f46afb344f4446af13d28'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.855480 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6f5916087f46afb344f4446af13d28'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.856370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6f5916087f46afb344f4446af13d28'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:48.874269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1300/2592] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:48 2022] DELETE /identity/v3/users/1c6f5916087f46afb344f4446af13d28 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:48.874430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:48.875724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:48.892345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666463d-758b-41fc-8f11-f42f8005e3e2 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:48.962329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:48.962970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:48.963273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:48.963537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] PATH_INFO: `/v3/projects/a3dd040eed3a4988afedbc12766d61cd` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:48.964839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3dd040eed3a4988afedbc12766d61cd)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:48.966813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.018036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3dd040eed3a4988afedbc12766d61cd', 'request_id': 'req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.026218 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3dd040eed3a4988afedbc12766d61cd', 'request_id': 'req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.035325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3dd040eed3a4988afedbc12766d61cd', 'request_id': 'req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.047960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] The token cache is being invalidate because project a3dd040eed3a4988afedbc12766d61cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:49.048383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.049431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.051442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1293/2593] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:48 2022] DELETE /identity/v3/projects/a3dd040eed3a4988afedbc12766d61cd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:49.055696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.057738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.071602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7c33e-06a3-412d-8fea-394a93ce1ecd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.134213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.134912 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.135187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.135513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] PATH_INFO: `/v3/users/6b55a63346f047abbd4c91922c0a169b` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.136757 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b55a63346f047abbd4c91922c0a169b)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.138964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.153689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:49.189236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.191173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.206282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c338cc2-aaf4-49a7-8537-3de23e4a4dd1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.207916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b55a63346f047abbd4c91922c0a169b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.211304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.211929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.212202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.212465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] PATH_INFO: `/v3/projects/5ab5df60b248465e81a9e5b119040417` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.213595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab5df60b248465e81a9e5b119040417)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.215500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.217809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b55a63346f047abbd4c91922c0a169b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.218886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b55a63346f047abbd4c91922c0a169b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.233751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b55a63346f047abbd4c91922c0a169b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.233971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:49.234977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b55a63346f047abbd4c91922c0a169b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.251796 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1301/2594] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:49 2022] DELETE /identity/v3/users/6b55a63346f047abbd4c91922c0a169b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:49.279255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab5df60b248465e81a9e5b119040417', 'request_id': 'req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.289169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab5df60b248465e81a9e5b119040417', 'request_id': 'req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.300496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab5df60b248465e81a9e5b119040417', 'request_id': 'req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.316678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] The token cache is being invalidate because project 5ab5df60b248465e81a9e5b119040417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:49.317365 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.317679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.319318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1294/2595] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:49 2022] DELETE /identity/v3/projects/5ab5df60b248465e81a9e5b119040417 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:49.527607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.529402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.615265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8c294-def9-413e-aef4-b75f8e5247dd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.623585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.625988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.680519 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e0b51-f669-48ce-aaa5-581ebcd7fae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.681228 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0b51-f669-48ce-aaa5-581ebcd7fae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.681491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0b51-f669-48ce-aaa5-581ebcd7fae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.681764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e0b51-f669-48ce-aaa5-581ebcd7fae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.683538 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1a2e0b51-f669-48ce-aaa5-581ebcd7fae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.712057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffddaa3-7cd9-4bdc-98a9-cb8c8d9f6f25 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.717434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.718982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.719242 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.719490 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] PATH_INFO: `/v3/projects/4c001a8a27204066b8df81b9db2980bd` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.720776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c001a8a27204066b8df81b9db2980bd)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.722850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.764108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:49.770421 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e0b51-f669-48ce-aaa5-581ebcd7fae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.772004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e0b51-f669-48ce-aaa5-581ebcd7fae9 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:49.784849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1302/2596] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:49.799429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c001a8a27204066b8df81b9db2980bd', 'request_id': 'req-f1d4e216-f370-4fa8-904d-a2633bf87520'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.808773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c001a8a27204066b8df81b9db2980bd', 'request_id': 'req-f1d4e216-f370-4fa8-904d-a2633bf87520'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.817018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c001a8a27204066b8df81b9db2980bd', 'request_id': 'req-f1d4e216-f370-4fa8-904d-a2633bf87520'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.831814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] The token cache is being invalidate because project 4c001a8a27204066b8df81b9db2980bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:49.832141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.832879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f1d4e216-f370-4fa8-904d-a2633bf87520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:49.834718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1295/2597] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:49 2022] DELETE /identity/v3/projects/4c001a8a27204066b8df81b9db2980bd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:49.840378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1d1f5-d8c9-4c4d-90c7-6c1b7b2e2616 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.840693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1d1f5-d8c9-4c4d-90c7-6c1b7b2e2616 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.840977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1d1f5-d8c9-4c4d-90c7-6c1b7b2e2616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.865754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8bc1d1f5-d8c9-4c4d-90c7-6c1b7b2e2616 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:49.867017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8bc1d1f5-d8c9-4c4d-90c7-6c1b7b2e2616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.931947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1303/2598] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:49.941518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69167f7b-68ea-4a1c-9627-b7f6c97dd134 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.941911 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69167f7b-68ea-4a1c-9627-b7f6c97dd134 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.942194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69167f7b-68ea-4a1c-9627-b7f6c97dd134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.963449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-69167f7b-68ea-4a1c-9627-b7f6c97dd134 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:49.964816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-69167f7b-68ea-4a1c-9627-b7f6c97dd134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:49.973081 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1296/2599] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:49.977378 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1d1f5-d8c9-4c4d-90c7-6c1b7b2e2616 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:49.988116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1d1f5-d8c9-4c4d-90c7-6c1b7b2e2616 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:49.992864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:49.993467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:49.993773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:49.994049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:49.994857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:49.995165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:49.996804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.009317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1304/2600] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:50.014197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69167f7b-68ea-4a1c-9627-b7f6c97dd134 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.024783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69167f7b-68ea-4a1c-9627-b7f6c97dd134 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.029448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97aa2-be2d-4847-8c11-e1ba3535d783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.030056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97aa2-be2d-4847-8c11-e1ba3535d783 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.030506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97aa2-be2d-4847-8c11-e1ba3535d783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.031043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97aa2-be2d-4847-8c11-e1ba3535d783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.031679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd97aa2-be2d-4847-8c11-e1ba3535d783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.033703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd97aa2-be2d-4847-8c11-e1ba3535d783 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.062282 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1297/2601] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:50.066772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.078966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a667d9d-71e7-4592-9fa8-3221f7789523 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.142686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a9a16-72d5-4e32-9d4b-08398006ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.143449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a9a16-72d5-4e32-9d4b-08398006ab4e admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.143832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a9a16-72d5-4e32-9d4b-08398006ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.144208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a9a16-72d5-4e32-9d4b-08398006ab4e admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.144999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a9a16-72d5-4e32-9d4b-08398006ab4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.147148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a9a16-72d5-4e32-9d4b-08398006ab4e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.175470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1305/2602] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:50.179823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97aa2-be2d-4847-8c11-e1ba3535d783 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.191556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97aa2-be2d-4847-8c11-e1ba3535d783 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.196634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9051cce-b47c-4c71-8c88-cd228438ad67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.197358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9051cce-b47c-4c71-8c88-cd228438ad67 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.197706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9051cce-b47c-4c71-8c88-cd228438ad67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.198044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9051cce-b47c-4c71-8c88-cd228438ad67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.198785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9051cce-b47c-4c71-8c88-cd228438ad67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.200652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9051cce-b47c-4c71-8c88-cd228438ad67 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.202970 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:50.216172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1298/2603] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:50.219865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a9a16-72d5-4e32-9d4b-08398006ab4e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.230237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a9a16-72d5-4e32-9d4b-08398006ab4e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.235931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ed0dd-63df-46e5-b46c-52992f120226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.236984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ed0dd-63df-46e5-b46c-52992f120226 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.237294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ed0dd-63df-46e5-b46c-52992f120226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.237572 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ed0dd-63df-46e5-b46c-52992f120226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.238248 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ed0dd-63df-46e5-b46c-52992f120226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.239892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ed0dd-63df-46e5-b46c-52992f120226 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.252534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1306/2604] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:50.256637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9051cce-b47c-4c71-8c88-cd228438ad67 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.267892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9051cce-b47c-4c71-8c88-cd228438ad67 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.273596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.274289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.274574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.274855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] PATH_INFO: `/v3/projects/521bdc54dd9d4fad9587bf9150553fa8/users/d596b76946fc4a62aaf1503c6e34ab0d/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.292044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=521bdc54dd9d4fad9587bf9150553fa8, user_id=d596b76946fc4a62aaf1503c6e34ab0d, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.294798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.312123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1299/2605] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:50 2022] PUT /identity/v3/projects/521bdc54dd9d4fad9587bf9150553fa8/users/d596b76946fc4a62aaf1503c6e34ab0d/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:50.323742 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f74a1-1d78-476b-87a9-654a76637b48 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.324025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f74a1-1d78-476b-87a9-654a76637b48 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.324293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f74a1-1d78-476b-87a9-654a76637b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.354815 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-181f74a1-1d78-476b-87a9-654a76637b48 None None] MFA Rules not processed for user `d596b76946fc4a62aaf1503c6e34ab0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:50.356065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-181f74a1-1d78-476b-87a9-654a76637b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.409732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:50.428755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1307/2606] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:50.440850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.442692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.504153 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2428f5-3838-4dfb-9b21-5d32cad1c82f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.507749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b1b0e-812b-4973-b672-bc6460168e10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.508360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b1b0e-812b-4973-b672-bc6460168e10 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.508673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b1b0e-812b-4973-b672-bc6460168e10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.508946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b1b0e-812b-4973-b672-bc6460168e10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.517769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b1b0e-812b-4973-b672-bc6460168e10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.519187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b1b0e-812b-4973-b672-bc6460168e10 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.528809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1300/2607] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:50.550598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f74a1-1d78-476b-87a9-654a76637b48 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.618362 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f74a1-1d78-476b-87a9-654a76637b48 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.623688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.624853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.625008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.625314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] PATH_INFO: `/v3/users/d596b76946fc4a62aaf1503c6e34ab0d` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.626483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d596b76946fc4a62aaf1503c6e34ab0d)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.628511 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.680216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd596b76946fc4a62aaf1503c6e34ab0d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.688483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd596b76946fc4a62aaf1503c6e34ab0d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.689237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd596b76946fc4a62aaf1503c6e34ab0d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.704067 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd596b76946fc4a62aaf1503c6e34ab0d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.705054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd596b76946fc4a62aaf1503c6e34ab0d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:50.720794 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1308/2608] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:50 2022] DELETE /identity/v3/users/d596b76946fc4a62aaf1503c6e34ab0d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:50.725065 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc22d7f-ac43-4faf-b3e8-a896e50c06f5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.725360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc22d7f-ac43-4faf-b3e8-a896e50c06f5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.725636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc22d7f-ac43-4faf-b3e8-a896e50c06f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.744979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1dc22d7f-ac43-4faf-b3e8-a896e50c06f5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:50.746201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1dc22d7f-ac43-4faf-b3e8-a896e50c06f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.808824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1301/2609] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:50.818636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:50.820183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:50.884033 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671049e-7f10-4ee0-9cec-d98a4bd60a5e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:50.887679 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca3fe1-ced0-49bd-89bc-4be9cc378446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:50.888298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca3fe1-ced0-49bd-89bc-4be9cc378446 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:50.888600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca3fe1-ced0-49bd-89bc-4be9cc378446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:50.888889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca3fe1-ced0-49bd-89bc-4be9cc378446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:50.899325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca3fe1-ced0-49bd-89bc-4be9cc378446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:50.900817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca3fe1-ced0-49bd-89bc-4be9cc378446 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:50.914273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1309/2610] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:51.318228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc22d7f-ac43-4faf-b3e8-a896e50c06f5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.319786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1dc22d7f-ac43-4faf-b3e8-a896e50c06f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.333839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc22d7f-ac43-4faf-b3e8-a896e50c06f5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.338718 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.339450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.339883 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.340145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] PATH_INFO: `/v3/projects/521bdc54dd9d4fad9587bf9150553fa8` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.342620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=521bdc54dd9d4fad9587bf9150553fa8)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.344756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.397524 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521bdc54dd9d4fad9587bf9150553fa8', 'request_id': 'req-bef9b8ca-42cf-459b-909b-2c927967f98b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.406936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521bdc54dd9d4fad9587bf9150553fa8', 'request_id': 'req-bef9b8ca-42cf-459b-909b-2c927967f98b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.413834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521bdc54dd9d4fad9587bf9150553fa8', 'request_id': 'req-bef9b8ca-42cf-459b-909b-2c927967f98b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.419979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:51.425789 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] The token cache is being invalidate because project 521bdc54dd9d4fad9587bf9150553fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:51.426361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.427565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bef9b8ca-42cf-459b-909b-2c927967f98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:51.429577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1302/2611] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:51 2022] DELETE /identity/v3/projects/521bdc54dd9d4fad9587bf9150553fa8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:51.434050 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a796e21-26c5-43d8-9e37-9425c7eacda9 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.434460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a796e21-26c5-43d8-9e37-9425c7eacda9 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.434837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a796e21-26c5-43d8-9e37-9425c7eacda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.456798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-7a796e21-26c5-43d8-9e37-9425c7eacda9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:51.458142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7a796e21-26c5-43d8-9e37-9425c7eacda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.467195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:51.528240 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1310/2612] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:51.538219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc557-aac6-4c60-b27a-9aa4ef55086f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.538737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc557-aac6-4c60-b27a-9aa4ef55086f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.538825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dc557-aac6-4c60-b27a-9aa4ef55086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.566543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-834dc557-aac6-4c60-b27a-9aa4ef55086f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:51.567921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-834dc557-aac6-4c60-b27a-9aa4ef55086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:51.580361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1303/2613] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:51.581877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a796e21-26c5-43d8-9e37-9425c7eacda9 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.592718 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a796e21-26c5-43d8-9e37-9425c7eacda9 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.598094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.598717 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.598997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.599271 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.599932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:51.600345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.602673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.619325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1311/2614] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:51.621059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dc557-aac6-4c60-b27a-9aa4ef55086f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.632668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dc557-aac6-4c60-b27a-9aa4ef55086f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.639062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b60d6f-a90c-4e0a-9ca6-5f6ee975543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.640982 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b60d6f-a90c-4e0a-9ca6-5f6ee975543d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.641468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b60d6f-a90c-4e0a-9ca6-5f6ee975543d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.641961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b60d6f-a90c-4e0a-9ca6-5f6ee975543d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.642608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b60d6f-a90c-4e0a-9ca6-5f6ee975543d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.644752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b60d6f-a90c-4e0a-9ca6-5f6ee975543d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.683478 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1304/2615] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:51.683932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.696961 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8084973-bdcb-423b-86e8-f5125c66e065 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.782826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968559-31af-446f-b29d-b1383d223544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.783455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968559-31af-446f-b29d-b1383d223544 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.783739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968559-31af-446f-b29d-b1383d223544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.784013 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d968559-31af-446f-b29d-b1383d223544 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.784745 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d968559-31af-446f-b29d-b1383d223544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.786752 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d968559-31af-446f-b29d-b1383d223544 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.828658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1312/2616] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:51 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:26:51.829989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b60d6f-a90c-4e0a-9ca6-5f6ee975543d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.841177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b60d6f-a90c-4e0a-9ca6-5f6ee975543d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.847762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a082-11d3-4874-a335-cd157a16618e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.848385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a082-11d3-4874-a335-cd157a16618e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.848714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a082-11d3-4874-a335-cd157a16618e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.848996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a082-11d3-4874-a335-cd157a16618e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.849631 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a082-11d3-4874-a335-cd157a16618e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.851247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a082-11d3-4874-a335-cd157a16618e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.863152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1305/2617] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:51.867434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968559-31af-446f-b29d-b1383d223544 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.880615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d968559-31af-446f-b29d-b1383d223544 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.885605 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c3984-42a7-47ee-a351-acf4dbcb8d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.886213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c3984-42a7-47ee-a351-acf4dbcb8d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.886486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c3984-42a7-47ee-a351-acf4dbcb8d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.886781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c3984-42a7-47ee-a351-acf4dbcb8d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:51.887426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c3984-42a7-47ee-a351-acf4dbcb8d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:51.889213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c3984-42a7-47ee-a351-acf4dbcb8d2f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:51.901188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1313/2618] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:51.905847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a082-11d3-4874-a335-cd157a16618e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:51.916426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a082-11d3-4874-a335-cd157a16618e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:51.922395 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:51.923212 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:51.923493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:51.923769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] PATH_INFO: `/v3/projects/f30c03a74d7f4482b62f04bb0bf94749/users/ff7e493a0ca1438c9cf21eed07b6bd1b/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.070451 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30c03a74d7f4482b62f04bb0bf94749, user_id=ff7e493a0ca1438c9cf21eed07b6bd1b, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.073742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.088510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1306/2619] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:51 2022] PUT /identity/v3/projects/f30c03a74d7f4482b62f04bb0bf94749/users/ff7e493a0ca1438c9cf21eed07b6bd1b/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:52.099937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b083-4a65-438f-a831-6bf45736805c None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.100415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b083-4a65-438f-a831-6bf45736805c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.101972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645b083-4a65-438f-a831-6bf45736805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.135145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a645b083-4a65-438f-a831-6bf45736805c None None] MFA Rules not processed for user `ff7e493a0ca1438c9cf21eed07b6bd1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:52.136396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a645b083-4a65-438f-a831-6bf45736805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:52.226806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1314/2620] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:52.239112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:52.240836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:52.308755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f1c3f-f3e2-4523-a11d-3bea8fb6efe0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:52.316008 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df6ea-763b-418a-a147-b153a5149b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:52.316743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df6ea-763b-418a-a147-b153a5149b54 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:52.317100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df6ea-763b-418a-a147-b153a5149b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:52.317473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df6ea-763b-418a-a147-b153a5149b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:52.327891 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173df6ea-763b-418a-a147-b153a5149b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:52.329450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173df6ea-763b-418a-a147-b153a5149b54 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:52.338866 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1307/2621] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:53.206254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645b083-4a65-438f-a831-6bf45736805c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.207812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a645b083-4a65-438f-a831-6bf45736805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.220652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:53.281869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645b083-4a65-438f-a831-6bf45736805c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.286660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.287289 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.287571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.287846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] PATH_INFO: `/v3/users/6e09978ed8b3490db5f92823768f1d15` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.289166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e09978ed8b3490db5f92823768f1d15)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.291104 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.348613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e09978ed8b3490db5f92823768f1d15'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.358565 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e09978ed8b3490db5f92823768f1d15'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.359738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e09978ed8b3490db5f92823768f1d15'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.378296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e09978ed8b3490db5f92823768f1d15'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.379023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-35f48b72-6245-47a1-84f6-1de1c117d004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e09978ed8b3490db5f92823768f1d15'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.402126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1315/2622] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:53 2022] DELETE /identity/v3/users/6e09978ed8b3490db5f92823768f1d15 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:53.588867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df6ea-763b-418a-a147-b153a5149b54 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.600522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df6ea-763b-418a-a147-b153a5149b54 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.658138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.658654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.658930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.659193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] PATH_INFO: `/v3/projects/1cb8f75600714e23bda1c8886bdb22f1` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.660273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cb8f75600714e23bda1c8886bdb22f1)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.662191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.707076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:53.716017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cb8f75600714e23bda1c8886bdb22f1', 'request_id': 'req-e9217826-b575-4cba-b451-7a725adc5820'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.731221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cb8f75600714e23bda1c8886bdb22f1', 'request_id': 'req-e9217826-b575-4cba-b451-7a725adc5820'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.731221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cb8f75600714e23bda1c8886bdb22f1', 'request_id': 'req-e9217826-b575-4cba-b451-7a725adc5820'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.742913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] The token cache is being invalidate because project 1cb8f75600714e23bda1c8886bdb22f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:53.743275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.744077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e9217826-b575-4cba-b451-7a725adc5820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:53.745842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1308/2623] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:53 2022] DELETE /identity/v3/projects/1cb8f75600714e23bda1c8886bdb22f1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:53.751424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6c362-2648-4c59-8044-66dd21e5b796 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.751607 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6c362-2648-4c59-8044-66dd21e5b796 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.751890 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6c362-2648-4c59-8044-66dd21e5b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.786630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e0e6c362-2648-4c59-8044-66dd21e5b796 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:53.787855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e0e6c362-2648-4c59-8044-66dd21e5b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.864600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1316/2624] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:53.872229 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44310fc-af9c-4bfe-86fe-b457d3169b7d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.872514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44310fc-af9c-4bfe-86fe-b457d3169b7d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.872926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44310fc-af9c-4bfe-86fe-b457d3169b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.894681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e44310fc-af9c-4bfe-86fe-b457d3169b7d None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:53.895890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e44310fc-af9c-4bfe-86fe-b457d3169b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.908699 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1309/2625] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:53.910155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6c362-2648-4c59-8044-66dd21e5b796 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.924689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6c362-2648-4c59-8044-66dd21e5b796 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.933965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.934748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.934965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.935234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.935917 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:53.936785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.939104 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.961707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1317/2626] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:53.966249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44310fc-af9c-4bfe-86fe-b457d3169b7d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.976599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44310fc-af9c-4bfe-86fe-b457d3169b7d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.982224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc53d87-3021-4d49-be72-29d479a19771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.982824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc53d87-3021-4d49-be72-29d479a19771 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.983636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc53d87-3021-4d49-be72-29d479a19771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.983958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc53d87-3021-4d49-be72-29d479a19771 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.984620 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc53d87-3021-4d49-be72-29d479a19771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.986563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc53d87-3021-4d49-be72-29d479a19771 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:54.024610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1310/2627] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:53 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:54.026264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:54.037657 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b6c64-00aa-4122-b1d4-d326ba649226 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:54.053777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:54.126777 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d217349-630f-4ae4-a3c0-44d671b65e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:54.128017 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d217349-630f-4ae4-a3c0-44d671b65e68 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.128818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d217349-630f-4ae4-a3c0-44d671b65e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.129076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d217349-630f-4ae4-a3c0-44d671b65e68 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.130207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d217349-630f-4ae4-a3c0-44d671b65e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:54.134132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d217349-630f-4ae4-a3c0-44d671b65e68 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:54.166892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1318/2628] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:54 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:54.171669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc53d87-3021-4d49-be72-29d479a19771 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:54.183098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc53d87-3021-4d49-be72-29d479a19771 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:54.191973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78d15a-e50d-4433-b1eb-7a23c30d0024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:54.193064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d15a-e50d-4433-b1eb-7a23c30d0024 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.193353 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d15a-e50d-4433-b1eb-7a23c30d0024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.193808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d15a-e50d-4433-b1eb-7a23c30d0024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.196245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78d15a-e50d-4433-b1eb-7a23c30d0024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:54.197897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78d15a-e50d-4433-b1eb-7a23c30d0024 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:54.208634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1311/2629] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:54.212803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d217349-630f-4ae4-a3c0-44d671b65e68 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:54.227916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d217349-630f-4ae4-a3c0-44d671b65e68 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:54.232809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433ffb8-3f4e-474f-b990-b1b4cb971a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:54.234195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ffb8-3f4e-474f-b990-b1b4cb971a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.234931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ffb8-3f4e-474f-b990-b1b4cb971a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.235084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433ffb8-3f4e-474f-b990-b1b4cb971a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.235803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433ffb8-3f4e-474f-b990-b1b4cb971a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:54.237530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433ffb8-3f4e-474f-b990-b1b4cb971a94 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:54.250203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1319/2630] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:54.254634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78d15a-e50d-4433-b1eb-7a23c30d0024 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:54.268809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78d15a-e50d-4433-b1eb-7a23c30d0024 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:54.279855 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:54.282040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.282040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.282040 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] PATH_INFO: `/v3/projects/0328689e0d4f4dbabdf06dff97dd8ad5/users/1c1bbd813918452fba618a78b4d0e2d6/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.298735 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0328689e0d4f4dbabdf06dff97dd8ad5, user_id=1c1bbd813918452fba618a78b4d0e2d6, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:54.301483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:54.315333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1312/2631] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:54 2022] PUT /identity/v3/projects/0328689e0d4f4dbabdf06dff97dd8ad5/users/1c1bbd813918452fba618a78b4d0e2d6/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:54.325222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad679c8c-1f5e-43ea-81ca-4c00d02ace52 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.325517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad679c8c-1f5e-43ea-81ca-4c00d02ace52 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.325793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad679c8c-1f5e-43ea-81ca-4c00d02ace52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.355902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-ad679c8c-1f5e-43ea-81ca-4c00d02ace52 None None] MFA Rules not processed for user `1c1bbd813918452fba618a78b4d0e2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:54.357167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ad679c8c-1f5e-43ea-81ca-4c00d02ace52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:54.428838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1320/2632] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:54.440517 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:54.442495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:54.522938 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57d924-a523-4ca5-9a31-987e1e6df954 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:54.531150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e525d-1386-41d6-840c-a4945cb78044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:54.531998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e525d-1386-41d6-840c-a4945cb78044 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.532389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e525d-1386-41d6-840c-a4945cb78044 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.532766 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e525d-1386-41d6-840c-a4945cb78044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.541453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e525d-1386-41d6-840c-a4945cb78044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:54.543091 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e525d-1386-41d6-840c-a4945cb78044 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:54.553224 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1313/2633] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:55.542778 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad679c8c-1f5e-43ea-81ca-4c00d02ace52 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:55.544335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ad679c8c-1f5e-43ea-81ca-4c00d02ace52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:55.610114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad679c8c-1f5e-43ea-81ca-4c00d02ace52 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:55.615100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:55.615711 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:55.615990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:55.616255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] PATH_INFO: `/v3/users/ff7e493a0ca1438c9cf21eed07b6bd1b` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:55.617481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7e493a0ca1438c9cf21eed07b6bd1b)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:55.619965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:55.650655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:55.674648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7e493a0ca1438c9cf21eed07b6bd1b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.683843 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7e493a0ca1438c9cf21eed07b6bd1b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.684686 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7e493a0ca1438c9cf21eed07b6bd1b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.699317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7e493a0ca1438c9cf21eed07b6bd1b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.700183 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7e493a0ca1438c9cf21eed07b6bd1b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:55.715977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1321/2634] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:55 2022] DELETE /identity/v3/users/ff7e493a0ca1438c9cf21eed07b6bd1b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:55.721120 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc595f5-e895-49fb-9bf0-f30d21139e79 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:55.721662 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc595f5-e895-49fb-9bf0-f30d21139e79 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:55.721781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc595f5-e895-49fb-9bf0-f30d21139e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:55.741879 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0dc595f5-e895-49fb-9bf0-f30d21139e79 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:55.743220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0dc595f5-e895-49fb-9bf0-f30d21139e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:55.799894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:55.819010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1314/2635] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:55.830333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:55.831773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:55.897803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce36feb-5a4d-400d-84c4-51bcda082632 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:55.901399 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae683d-5224-43d6-a738-24b7e2c91713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:55.901992 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae683d-5224-43d6-a738-24b7e2c91713 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:55.902265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae683d-5224-43d6-a738-24b7e2c91713 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:55.902530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae683d-5224-43d6-a738-24b7e2c91713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:55.911632 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae683d-5224-43d6-a738-24b7e2c91713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:55.913106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae683d-5224-43d6-a738-24b7e2c91713 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:55.922582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1322/2636] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:56.223644 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc595f5-e895-49fb-9bf0-f30d21139e79 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.225888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0dc595f5-e895-49fb-9bf0-f30d21139e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:56.245328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc595f5-e895-49fb-9bf0-f30d21139e79 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.250987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.251755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.252129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.252626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] PATH_INFO: `/v3/projects/f30c03a74d7f4482b62f04bb0bf94749` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.254418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f30c03a74d7f4482b62f04bb0bf94749)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.257160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.306876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f30c03a74d7f4482b62f04bb0bf94749', 'request_id': 'req-38181afc-9257-4575-b1cd-ca4766d3a6fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.314077 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f30c03a74d7f4482b62f04bb0bf94749', 'request_id': 'req-38181afc-9257-4575-b1cd-ca4766d3a6fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.320749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f30c03a74d7f4482b62f04bb0bf94749', 'request_id': 'req-38181afc-9257-4575-b1cd-ca4766d3a6fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.331551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] The token cache is being invalidate because project f30c03a74d7f4482b62f04bb0bf94749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:56.331842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.332480 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38181afc-9257-4575-b1cd-ca4766d3a6fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:56.334074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1315/2637] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:56 2022] DELETE /identity/v3/projects/f30c03a74d7f4482b62f04bb0bf94749 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:56.338993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb0b13-08ac-4382-b615-74f95f05ad4f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.339276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb0b13-08ac-4382-b615-74f95f05ad4f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.339547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb0b13-08ac-4382-b615-74f95f05ad4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.359598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-0efb0b13-08ac-4382-b615-74f95f05ad4f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:56.360819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0efb0b13-08ac-4382-b615-74f95f05ad4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:56.422591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1323/2638] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:56.430477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6db115-2254-4b44-a942-1b4a6e432e44 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.430760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6db115-2254-4b44-a942-1b4a6e432e44 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.431032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6db115-2254-4b44-a942-1b4a6e432e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.451227 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0b6db115-2254-4b44-a942-1b4a6e432e44 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:56.453150 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0b6db115-2254-4b44-a942-1b4a6e432e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:56.461190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1316/2639] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:56.465417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb0b13-08ac-4382-b615-74f95f05ad4f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.477130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb0b13-08ac-4382-b615-74f95f05ad4f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.481712 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.482300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.482577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.482844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.483652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:56.483966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.485621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.497407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1324/2640] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:56.502134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6db115-2254-4b44-a942-1b4a6e432e44 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.511564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6db115-2254-4b44-a942-1b4a6e432e44 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.517592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed58fc-3d37-4f1e-b310-d997c47de980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.518186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed58fc-3d37-4f1e-b310-d997c47de980 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.518461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed58fc-3d37-4f1e-b310-d997c47de980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.518733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed58fc-3d37-4f1e-b310-d997c47de980 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.519369 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed58fc-3d37-4f1e-b310-d997c47de980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.521287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed58fc-3d37-4f1e-b310-d997c47de980 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.551087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1317/2641] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:56 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:56.555396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.566226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0361735-762b-43af-9890-9154fce19dcd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.599435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:56.628129 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc78a7-5fea-41b1-85bd-99969b72b8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.628836 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc78a7-5fea-41b1-85bd-99969b72b8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.629117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc78a7-5fea-41b1-85bd-99969b72b8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.629383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc78a7-5fea-41b1-85bd-99969b72b8e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.630015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc78a7-5fea-41b1-85bd-99969b72b8e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.631931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc78a7-5fea-41b1-85bd-99969b72b8e5 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.653853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1325/2642] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:56 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:56.658513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed58fc-3d37-4f1e-b310-d997c47de980 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.669324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed58fc-3d37-4f1e-b310-d997c47de980 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.673935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cfc20-0199-429a-9960-21261f173d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.674543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cfc20-0199-429a-9960-21261f173d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.674806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cfc20-0199-429a-9960-21261f173d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.675076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cfc20-0199-429a-9960-21261f173d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.675715 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cfc20-0199-429a-9960-21261f173d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.677332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cfc20-0199-429a-9960-21261f173d49 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.688094 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1318/2643] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:56.692150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc78a7-5fea-41b1-85bd-99969b72b8e5 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.702915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc78a7-5fea-41b1-85bd-99969b72b8e5 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.707528 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae573337-7fc6-4afb-a021-6b403049cb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.708121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae573337-7fc6-4afb-a021-6b403049cb77 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.708402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae573337-7fc6-4afb-a021-6b403049cb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.708713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae573337-7fc6-4afb-a021-6b403049cb77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.709357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae573337-7fc6-4afb-a021-6b403049cb77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.710973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae573337-7fc6-4afb-a021-6b403049cb77 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.721846 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1326/2644] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:56.726013 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cfc20-0199-429a-9960-21261f173d49 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.736087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cfc20-0199-429a-9960-21261f173d49 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.740697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.741332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.741607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.741875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] PATH_INFO: `/v3/projects/614275b03a454a548ce65fe8264b49a2/users/26a267ca9b6f443ebdcd39ea1a5ed537/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.757694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614275b03a454a548ce65fe8264b49a2, user_id=26a267ca9b6f443ebdcd39ea1a5ed537, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.761165 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.774596 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1319/2645] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:56 2022] PUT /identity/v3/projects/614275b03a454a548ce65fe8264b49a2/users/26a267ca9b6f443ebdcd39ea1a5ed537/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:56.786047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042770f8-a3b4-472e-b75e-72fbf91adbd2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.786339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042770f8-a3b4-472e-b75e-72fbf91adbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.786638 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042770f8-a3b4-472e-b75e-72fbf91adbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.815790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-042770f8-a3b4-472e-b75e-72fbf91adbd2 None None] MFA Rules not processed for user `26a267ca9b6f443ebdcd39ea1a5ed537`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:56.817189 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-042770f8-a3b4-472e-b75e-72fbf91adbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:56.881026 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1327/2646] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:56.893330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:56.895103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:56.958598 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc55e9-6d33-417b-acb6-c24364d73842 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:56.962333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfd7b2-80af-48f9-919c-8e4fa561fe0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:56.962945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd7b2-80af-48f9-919c-8e4fa561fe0a service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:56.963225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd7b2-80af-48f9-919c-8e4fa561fe0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:56.963495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd7b2-80af-48f9-919c-8e4fa561fe0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:56.974796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfd7b2-80af-48f9-919c-8e4fa561fe0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:56.976315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfd7b2-80af-48f9-919c-8e4fa561fe0a service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:56.987045 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1320/2647] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:57.300493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042770f8-a3b4-472e-b75e-72fbf91adbd2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:57.302071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-042770f8-a3b4-472e-b75e-72fbf91adbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:57.373230 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042770f8-a3b4-472e-b75e-72fbf91adbd2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:57.378785 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:57.379496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:57.379820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:57.380094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] PATH_INFO: `/v3/users/1c1bbd813918452fba618a78b4d0e2d6` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:57.381413 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1bbd813918452fba618a78b4d0e2d6)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:57.383461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:57.429255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:57.431997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1bbd813918452fba618a78b4d0e2d6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.440814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1bbd813918452fba618a78b4d0e2d6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.441477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1bbd813918452fba618a78b4d0e2d6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.454296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1bbd813918452fba618a78b4d0e2d6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.454940 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1bbd813918452fba618a78b4d0e2d6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:57.467212 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1328/2648] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:26:57 2022] DELETE /identity/v3/users/1c1bbd813918452fba618a78b4d0e2d6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:57.472303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb73a4-b3c7-4f4f-9c79-d2d50c7ff0bb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:57.472628 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb73a4-b3c7-4f4f-9c79-d2d50c7ff0bb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:57.472926 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb73a4-b3c7-4f4f-9c79-d2d50c7ff0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:57.492174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fdeb73a4-b3c7-4f4f-9c79-d2d50c7ff0bb None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:57.493430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fdeb73a4-b3c7-4f4f-9c79-d2d50c7ff0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:57.498445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:57.554746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1321/2649] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:57.564955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:57.566415 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:57.622849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255b2c-044e-4230-ade1-37c862e092f6 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:57.627025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d386b12-0bf8-4d1f-ab86-9ef3eb065b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:57.627624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d386b12-0bf8-4d1f-ab86-9ef3eb065b6c service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:57.627899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d386b12-0bf8-4d1f-ab86-9ef3eb065b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:57.628194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d386b12-0bf8-4d1f-ab86-9ef3eb065b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:57.637258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d386b12-0bf8-4d1f-ab86-9ef3eb065b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:57.638699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d386b12-0bf8-4d1f-ab86-9ef3eb065b6c service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:57.648656 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1329/2650] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:57.926140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb73a4-b3c7-4f4f-9c79-d2d50c7ff0bb None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:57.927603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fdeb73a4-b3c7-4f4f-9c79-d2d50c7ff0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:57.942024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb73a4-b3c7-4f4f-9c79-d2d50c7ff0bb None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:57.946827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:57.947495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:57.947834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:57.948181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] PATH_INFO: `/v3/projects/0328689e0d4f4dbabdf06dff97dd8ad5` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:57.949423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0328689e0d4f4dbabdf06dff97dd8ad5)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:57.951436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.001050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0328689e0d4f4dbabdf06dff97dd8ad5', 'request_id': 'req-68deb9aa-f091-4393-8217-5ae0fff61191'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.008307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0328689e0d4f4dbabdf06dff97dd8ad5', 'request_id': 'req-68deb9aa-f091-4393-8217-5ae0fff61191'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.014748 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0328689e0d4f4dbabdf06dff97dd8ad5', 'request_id': 'req-68deb9aa-f091-4393-8217-5ae0fff61191'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.025250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] The token cache is being invalidate because project 0328689e0d4f4dbabdf06dff97dd8ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:26:58.025537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.026167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-68deb9aa-f091-4393-8217-5ae0fff61191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:26:58.027626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1322/2651] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:26:57 2022] DELETE /identity/v3/projects/0328689e0d4f4dbabdf06dff97dd8ad5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:26:58.033123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f527-4c3a-4948-b6ba-7d5663030b9f None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.033418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f527-4c3a-4948-b6ba-7d5663030b9f None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.033695 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f527-4c3a-4948-b6ba-7d5663030b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.053346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-a415f527-4c3a-4948-b6ba-7d5663030b9f None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:58.054556 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a415f527-4c3a-4948-b6ba-7d5663030b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.117433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1330/2652] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:58.126638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acb518-574b-4370-bae9-fa9968d7315c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.126946 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acb518-574b-4370-bae9-fa9968d7315c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.127184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41acb518-574b-4370-bae9-fa9968d7315c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.148314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-41acb518-574b-4370-bae9-fa9968d7315c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:58.149572 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-41acb518-574b-4370-bae9-fa9968d7315c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.158603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1323/2653] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:58.162894 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415f527-4c3a-4948-b6ba-7d5663030b9f None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.174413 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415f527-4c3a-4948-b6ba-7d5663030b9f None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.180313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.181046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.181430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.181819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.182564 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:26:58.182965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.184828 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.198914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1331/2654] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:26:58.203296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acb518-574b-4370-bae9-fa9968d7315c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.213265 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41acb518-574b-4370-bae9-fa9968d7315c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.218520 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902ca23-fd11-4dbe-b7d2-0f38bdcb077d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.219277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902ca23-fd11-4dbe-b7d2-0f38bdcb077d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.219558 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902ca23-fd11-4dbe-b7d2-0f38bdcb077d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.219830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902ca23-fd11-4dbe-b7d2-0f38bdcb077d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.220673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902ca23-fd11-4dbe-b7d2-0f38bdcb077d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.222594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902ca23-fd11-4dbe-b7d2-0f38bdcb077d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.254035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1324/2655] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:58.258442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.269467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b31c7-9eb9-4cf5-8958-6c43875b9def admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.319478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:58.333754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac078d43-5b5b-4e41-baf3-b11c107f645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.334697 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac078d43-5b5b-4e41-baf3-b11c107f645f admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.335161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac078d43-5b5b-4e41-baf3-b11c107f645f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.335537 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac078d43-5b5b-4e41-baf3-b11c107f645f admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.336222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac078d43-5b5b-4e41-baf3-b11c107f645f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.338285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac078d43-5b5b-4e41-baf3-b11c107f645f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.365484 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1332/2656] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/users => generated 279 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:58.369484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902ca23-fd11-4dbe-b7d2-0f38bdcb077d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.382274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902ca23-fd11-4dbe-b7d2-0f38bdcb077d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.387803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cefce-1321-4bd9-9d40-157662dc341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.388438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cefce-1321-4bd9-9d40-157662dc341e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.388751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cefce-1321-4bd9-9d40-157662dc341e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.389018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cefce-1321-4bd9-9d40-157662dc341e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.389697 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cefce-1321-4bd9-9d40-157662dc341e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.391637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cefce-1321-4bd9-9d40-157662dc341e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.402346 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1325/2657] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:26:58.407514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac078d43-5b5b-4e41-baf3-b11c107f645f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.418072 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac078d43-5b5b-4e41-baf3-b11c107f645f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.425034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.425662 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.426009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.426354 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.427074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.428783 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.447188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1333/2658] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:58.453188 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cefce-1321-4bd9-9d40-157662dc341e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.463732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cefce-1321-4bd9-9d40-157662dc341e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.467888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.469749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d8300-d3ca-4bfd-92aa-dc120a142e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.470425 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d8300-d3ca-4bfd-92aa-dc120a142e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.470709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d8300-d3ca-4bfd-92aa-dc120a142e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.470829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.470981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d8300-d3ca-4bfd-92aa-dc120a142e60 admin admin] PATH_INFO: `/v3/projects/0a82469e2dc040369bf1fe7def435ca2/users/ba445134e68241ec92f3c3c40127fb5b/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.488443 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d8300-d3ca-4bfd-92aa-dc120a142e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a82469e2dc040369bf1fe7def435ca2, user_id=ba445134e68241ec92f3c3c40127fb5b, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.491601 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d8300-d3ca-4bfd-92aa-dc120a142e60 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.506307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1326/2659] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:58 2022] PUT /identity/v3/projects/0a82469e2dc040369bf1fe7def435ca2/users/ba445134e68241ec92f3c3c40127fb5b/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:58.517448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db6c22-13cb-46b5-93d9-02efc5f23a93 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.517851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db6c22-13cb-46b5-93d9-02efc5f23a93 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.518171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db6c22-13cb-46b5-93d9-02efc5f23a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.543789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3495a3-2ef6-4370-b06c-b1bbc11d6315 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.552324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c9db6c22-13cb-46b5-93d9-02efc5f23a93 None None] MFA Rules not processed for user `ba445134e68241ec92f3c3c40127fb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:58.553845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c9db6c22-13cb-46b5-93d9-02efc5f23a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.574623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:58.603513 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.604152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.604413 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.605227 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.607467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:58.644487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1327/2660] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:26:58.645744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db6c22-13cb-46b5-93d9-02efc5f23a93 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.682191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.683645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.704793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1334/2661] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:58.714672 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db6c22-13cb-46b5-93d9-02efc5f23a93 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.721090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16041dd5-fc72-4b6c-9ac8-529e99270293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.721820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16041dd5-fc72-4b6c-9ac8-529e99270293 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.722100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16041dd5-fc72-4b6c-9ac8-529e99270293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.722372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16041dd5-fc72-4b6c-9ac8-529e99270293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.723016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16041dd5-fc72-4b6c-9ac8-529e99270293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.724962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16041dd5-fc72-4b6c-9ac8-529e99270293 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.762347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1328/2662] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:58.767234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.778579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598b61e-30d8-47dc-a4cb-4224e2aebb8e service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.820624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:58.854614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14842b-55ed-48e2-8d0e-a828df15adc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.855606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14842b-55ed-48e2-8d0e-a828df15adc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.856111 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14842b-55ed-48e2-8d0e-a828df15adc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.856930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14842b-55ed-48e2-8d0e-a828df15adc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.859256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14842b-55ed-48e2-8d0e-a828df15adc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.862955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14842b-55ed-48e2-8d0e-a828df15adc0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.910615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1335/2663] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:26:58 2022] POST /identity/v3/users => generated 284 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:26:58.911376 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16041dd5-fc72-4b6c-9ac8-529e99270293 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.923627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16041dd5-fc72-4b6c-9ac8-529e99270293 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.933977 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cc344-bf8b-423e-9fa6-cdcbd52f635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.934687 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cc344-bf8b-423e-9fa6-cdcbd52f635b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.934975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cc344-bf8b-423e-9fa6-cdcbd52f635b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.935245 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cc344-bf8b-423e-9fa6-cdcbd52f635b admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.937025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cc344-bf8b-423e-9fa6-cdcbd52f635b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.938329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cc344-bf8b-423e-9fa6-cdcbd52f635b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:58.955014 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14842b-55ed-48e2-8d0e-a828df15adc0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:58.955428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1329/2664] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:58.965826 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14842b-55ed-48e2-8d0e-a828df15adc0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:58.972275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8cbfa-0673-4411-a86c-d33ddaeced9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:58.973027 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8cbfa-0673-4411-a86c-d33ddaeced9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:58.973357 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8cbfa-0673-4411-a86c-d33ddaeced9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:58.973902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8cbfa-0673-4411-a86c-d33ddaeced9e admin admin] PATH_INFO: `/v3/projects/87c23e57dab14f98aff38d4d792856f7/users/26c8473756634a348f8cf95cc16188f7/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:58.990275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8cbfa-0673-4411-a86c-d33ddaeced9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c23e57dab14f98aff38d4d792856f7, user_id=26c8473756634a348f8cf95cc16188f7, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:58.993249 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8cbfa-0673-4411-a86c-d33ddaeced9e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.008236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1336/2665] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:58 2022] PUT /identity/v3/projects/87c23e57dab14f98aff38d4d792856f7/users/26c8473756634a348f8cf95cc16188f7/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:59.012256 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cc344-bf8b-423e-9fa6-cdcbd52f635b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.022802 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cc344-bf8b-423e-9fa6-cdcbd52f635b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.083985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523332a1-d863-470d-8876-cd18e955ad02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.084615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523332a1-d863-470d-8876-cd18e955ad02 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.084899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523332a1-d863-470d-8876-cd18e955ad02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.085167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523332a1-d863-470d-8876-cd18e955ad02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.085805 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523332a1-d863-470d-8876-cd18e955ad02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.087398 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523332a1-d863-470d-8876-cd18e955ad02 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.102174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1330/2666] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:26:59.103645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8cbfa-0673-4411-a86c-d33ddaeced9e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.114324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8cbfa-0673-4411-a86c-d33ddaeced9e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.119814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.121571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.122237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.122907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] PATH_INFO: `/v3/projects/87c23e57dab14f98aff38d4d792856f7/users/26c8473756634a348f8cf95cc16188f7/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.126181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c23e57dab14f98aff38d4d792856f7, user_id=26c8473756634a348f8cf95cc16188f7, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.129246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.144819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1337/2667] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:26:59 2022] PUT /identity/v3/projects/87c23e57dab14f98aff38d4d792856f7/users/26c8473756634a348f8cf95cc16188f7/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:26:59.156589 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880ad6c-acf2-465a-b12f-7cc05a19c04c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.156884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880ad6c-acf2-465a-b12f-7cc05a19c04c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.157249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880ad6c-acf2-465a-b12f-7cc05a19c04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.185865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b880ad6c-acf2-465a-b12f-7cc05a19c04c None None] MFA Rules not processed for user `26c8473756634a348f8cf95cc16188f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:59.187903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b880ad6c-acf2-465a-b12f-7cc05a19c04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:59.208332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:59.268010 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1331/2668] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:26:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:59.281971 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.284087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:59.347483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306477fb-2ff0-4e1f-a369-0ca5ca898b13 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.351130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e354d9-7d4f-410c-89d4-fdd24d96ea45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.351729 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e354d9-7d4f-410c-89d4-fdd24d96ea45 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.352006 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e354d9-7d4f-410c-89d4-fdd24d96ea45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.352273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e354d9-7d4f-410c-89d4-fdd24d96ea45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.361621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e354d9-7d4f-410c-89d4-fdd24d96ea45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.363068 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e354d9-7d4f-410c-89d4-fdd24d96ea45 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.372654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1338/2669] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:59.452630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880ad6c-acf2-465a-b12f-7cc05a19c04c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:59.462455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880ad6c-acf2-465a-b12f-7cc05a19c04c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:59.466772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340125a-c2ae-4d03-938b-3dfd53d25265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:59.467406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340125a-c2ae-4d03-938b-3dfd53d25265 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:59.467701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340125a-c2ae-4d03-938b-3dfd53d25265 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:59.468272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340125a-c2ae-4d03-938b-3dfd53d25265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:59.526775 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340125a-c2ae-4d03-938b-3dfd53d25265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:59.528250 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340125a-c2ae-4d03-938b-3dfd53d25265 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:59.537463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1332/2670] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:09.368343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=54,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:09.533418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=48 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:16.721744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e354d9-7d4f-410c-89d4-fdd24d96ea45 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:16.786115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e354d9-7d4f-410c-89d4-fdd24d96ea45 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:16.790764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:16.791481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:16.791827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:16.792101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] PATH_INFO: `/v3/users/ba445134e68241ec92f3c3c40127fb5b` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:16.793709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba445134e68241ec92f3c3c40127fb5b)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:16.796636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:16.847536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba445134e68241ec92f3c3c40127fb5b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.854781 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba445134e68241ec92f3c3c40127fb5b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.855486 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba445134e68241ec92f3c3c40127fb5b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.867557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba445134e68241ec92f3c3c40127fb5b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.868208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba445134e68241ec92f3c3c40127fb5b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:16.881577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1339/2671] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:16 2022] DELETE /identity/v3/users/ba445134e68241ec92f3c3c40127fb5b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:16.886007 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a67aab-d3fc-4792-9b61-d7a575321796 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:16.886514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a67aab-d3fc-4792-9b61-d7a575321796 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:16.886630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a67aab-d3fc-4792-9b61-d7a575321796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:16.908178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c5a67aab-d3fc-4792-9b61-d7a575321796 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:16.909417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c5a67aab-d3fc-4792-9b61-d7a575321796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:16.969402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1333/2672] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:16.979208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:16.980648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:17.043840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b45289-9b25-4347-a1f7-305587767c09 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:17.047750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d14e-8633-4408-837d-507259e74739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:17.048353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d14e-8633-4408-837d-507259e74739 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:17.048669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d14e-8633-4408-837d-507259e74739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:17.048941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64d14e-8633-4408-837d-507259e74739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:17.058442 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64d14e-8633-4408-837d-507259e74739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:17.059991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64d14e-8633-4408-837d-507259e74739 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:17.071082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1340/2673] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:17.348277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a67aab-d3fc-4792-9b61-d7a575321796 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:17.349716 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c5a67aab-d3fc-4792-9b61-d7a575321796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:17.362671 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a67aab-d3fc-4792-9b61-d7a575321796 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:17.367702 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:17.368342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:17.368649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:17.368924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] PATH_INFO: `/v3/projects/0a82469e2dc040369bf1fe7def435ca2` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:17.370191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a82469e2dc040369bf1fe7def435ca2)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:17.372226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:17.423863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a82469e2dc040369bf1fe7def435ca2', 'request_id': 'req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.431124 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a82469e2dc040369bf1fe7def435ca2', 'request_id': 'req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.437417 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a82469e2dc040369bf1fe7def435ca2', 'request_id': 'req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.449357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] The token cache is being invalidate because project 0a82469e2dc040369bf1fe7def435ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:17.449644 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.450294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.452315 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1334/2674] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:17 2022] DELETE /identity/v3/projects/0a82469e2dc040369bf1fe7def435ca2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:17.456368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d14e-8633-4408-837d-507259e74739 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:17.466195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64d14e-8633-4408-837d-507259e74739 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:17.487466 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:17.529288 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:17.530950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:17.531235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:17.531502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] PATH_INFO: `/v3/users/26c8473756634a348f8cf95cc16188f7` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:17.532669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c8473756634a348f8cf95cc16188f7)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:17.534595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:17.581764 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c8473756634a348f8cf95cc16188f7'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.588889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c8473756634a348f8cf95cc16188f7'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.589604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c8473756634a348f8cf95cc16188f7'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.602356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c8473756634a348f8cf95cc16188f7'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.603382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1291eab7-eb9c-430a-b4c7-7660ad490c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c8473756634a348f8cf95cc16188f7'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:17.616209 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1341/2675] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:17 2022] DELETE /identity/v3/users/26c8473756634a348f8cf95cc16188f7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:17.917470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:17.919289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:17.938494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422e5f1-b451-4520-9ff8-b3a54db8a3c1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.004829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.005347 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.005624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.005895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] PATH_INFO: `/v3/projects/87c23e57dab14f98aff38d4d792856f7` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.007129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c23e57dab14f98aff38d4d792856f7)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.009156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.041872 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:18.059623 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c23e57dab14f98aff38d4d792856f7', 'request_id': 'req-c13e7b4f-4b25-4751-b851-14c11c185598'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.073615 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c23e57dab14f98aff38d4d792856f7', 'request_id': 'req-c13e7b4f-4b25-4751-b851-14c11c185598'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.081036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c23e57dab14f98aff38d4d792856f7', 'request_id': 'req-c13e7b4f-4b25-4751-b851-14c11c185598'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.092308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] The token cache is being invalidate because project 87c23e57dab14f98aff38d4d792856f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:18.092635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.093339 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c13e7b4f-4b25-4751-b851-14c11c185598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:18.094845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1335/2676] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:17 2022] DELETE /identity/v3/projects/87c23e57dab14f98aff38d4d792856f7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:18.100784 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c402064-52ed-487c-8a11-112f0814b714 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.101309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c402064-52ed-487c-8a11-112f0814b714 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.101309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c402064-52ed-487c-8a11-112f0814b714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.121968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8c402064-52ed-487c-8a11-112f0814b714 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:18.123827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8c402064-52ed-487c-8a11-112f0814b714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:18.188336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1342/2677] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:18.196177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb224ae-87c1-4125-8412-4b7d107b3fa4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.196683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb224ae-87c1-4125-8412-4b7d107b3fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.197055 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb224ae-87c1-4125-8412-4b7d107b3fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.217569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7fb224ae-87c1-4125-8412-4b7d107b3fa4 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:18.218853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7fb224ae-87c1-4125-8412-4b7d107b3fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:18.226812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1336/2678] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:18.231302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c402064-52ed-487c-8a11-112f0814b714 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.241738 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c402064-52ed-487c-8a11-112f0814b714 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.246335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.246982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.247393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.247544 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.248891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:18.249203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.250829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.263097 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1343/2679] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:18.267487 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb224ae-87c1-4125-8412-4b7d107b3fa4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.277413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb224ae-87c1-4125-8412-4b7d107b3fa4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.282996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112a48c-e252-41e4-b4e8-e0426169765d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.283594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112a48c-e252-41e4-b4e8-e0426169765d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.283871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112a48c-e252-41e4-b4e8-e0426169765d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.284202 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112a48c-e252-41e4-b4e8-e0426169765d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.284882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112a48c-e252-41e4-b4e8-e0426169765d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.287070 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112a48c-e252-41e4-b4e8-e0426169765d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.318613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1337/2680] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:18 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:18.322936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.333555 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5b875-fdb6-4738-990c-6b4708d9c62b admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.370428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:18.402165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924144e5-8b65-4774-835c-a827c46eba24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.402773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924144e5-8b65-4774-835c-a827c46eba24 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.403089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924144e5-8b65-4774-835c-a827c46eba24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.403423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924144e5-8b65-4774-835c-a827c46eba24 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.404913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924144e5-8b65-4774-835c-a827c46eba24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.407182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924144e5-8b65-4774-835c-a827c46eba24 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.432306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1344/2681] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:18 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:18.433544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112a48c-e252-41e4-b4e8-e0426169765d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.443884 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112a48c-e252-41e4-b4e8-e0426169765d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.448519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1671426-7aa1-47d8-bcda-004a48fc5e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.449229 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1671426-7aa1-47d8-bcda-004a48fc5e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.449502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1671426-7aa1-47d8-bcda-004a48fc5e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.449767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1671426-7aa1-47d8-bcda-004a48fc5e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.450404 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1671426-7aa1-47d8-bcda-004a48fc5e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.452072 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1671426-7aa1-47d8-bcda-004a48fc5e02 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.462336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1338/2682] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:18.466356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924144e5-8b65-4774-835c-a827c46eba24 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.477566 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924144e5-8b65-4774-835c-a827c46eba24 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.483150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d744f0f-784e-4566-8b3d-77159a4316c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.483750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744f0f-784e-4566-8b3d-77159a4316c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.484479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744f0f-784e-4566-8b3d-77159a4316c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.484842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744f0f-784e-4566-8b3d-77159a4316c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.485773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d744f0f-784e-4566-8b3d-77159a4316c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.488055 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d744f0f-784e-4566-8b3d-77159a4316c5 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.500807 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1345/2683] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:18.505527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1671426-7aa1-47d8-bcda-004a48fc5e02 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.516011 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1671426-7aa1-47d8-bcda-004a48fc5e02 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.520893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d0de7-9824-499e-bd41-c4c98c6ed7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.521521 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d0de7-9824-499e-bd41-c4c98c6ed7be admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.521830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d0de7-9824-499e-bd41-c4c98c6ed7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.522322 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d0de7-9824-499e-bd41-c4c98c6ed7be admin admin] PATH_INFO: `/v3/projects/ede2fb72ba184682878c1fe2edb1ba56/users/5c510ba75c114429a140d69f6e003130/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.540180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d0de7-9824-499e-bd41-c4c98c6ed7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede2fb72ba184682878c1fe2edb1ba56, user_id=5c510ba75c114429a140d69f6e003130, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.542848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d0de7-9824-499e-bd41-c4c98c6ed7be admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.557359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1339/2684] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:18 2022] PUT /identity/v3/projects/ede2fb72ba184682878c1fe2edb1ba56/users/5c510ba75c114429a140d69f6e003130/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:18.568817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c841eed-4664-4e4d-800c-a2203d0f3a42 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.569107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c841eed-4664-4e4d-800c-a2203d0f3a42 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.569508 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c841eed-4664-4e4d-800c-a2203d0f3a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.597716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-2c841eed-4664-4e4d-800c-a2203d0f3a42 None None] MFA Rules not processed for user `5c510ba75c114429a140d69f6e003130`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:18.598932 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2c841eed-4664-4e4d-800c-a2203d0f3a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:18.670437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1346/2685] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:18.676185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d0de7-9824-499e-bd41-c4c98c6ed7be admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.738796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d0de7-9824-499e-bd41-c4c98c6ed7be admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.743633 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8ff59-0cc3-492c-92d2-b3b3a16522af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.744232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ff59-0cc3-492c-92d2-b3b3a16522af admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.744511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ff59-0cc3-492c-92d2-b3b3a16522af admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.744808 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ff59-0cc3-492c-92d2-b3b3a16522af admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.745433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8ff59-0cc3-492c-92d2-b3b3a16522af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.747498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8ff59-0cc3-492c-92d2-b3b3a16522af admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.777906 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1340/2686] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:18 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:18.784499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c841eed-4664-4e4d-800c-a2203d0f3a42 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.797109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c841eed-4664-4e4d-800c-a2203d0f3a42 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.855301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4784fbd-239b-43b5-831a-06b578dc94c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.855904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4784fbd-239b-43b5-831a-06b578dc94c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.856179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4784fbd-239b-43b5-831a-06b578dc94c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.856448 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4784fbd-239b-43b5-831a-06b578dc94c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.857108 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4784fbd-239b-43b5-831a-06b578dc94c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.859076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4784fbd-239b-43b5-831a-06b578dc94c9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.882604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1347/2687] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:18 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:18.886595 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8ff59-0cc3-492c-92d2-b3b3a16522af admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.898244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8ff59-0cc3-492c-92d2-b3b3a16522af admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.903003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518afd9-fff4-4933-aad7-b208f29f35f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.903611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518afd9-fff4-4933-aad7-b208f29f35f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.903881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518afd9-fff4-4933-aad7-b208f29f35f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.905511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518afd9-fff4-4933-aad7-b208f29f35f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.905960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518afd9-fff4-4933-aad7-b208f29f35f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.907574 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518afd9-fff4-4933-aad7-b208f29f35f3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.928375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1341/2688] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:18.932524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4784fbd-239b-43b5-831a-06b578dc94c9 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.942963 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4784fbd-239b-43b5-831a-06b578dc94c9 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:18.947878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be21469-ef9e-4c11-8a21-441df5f47fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:18.948522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be21469-ef9e-4c11-8a21-441df5f47fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:18.948847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be21469-ef9e-4c11-8a21-441df5f47fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:18.949087 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be21469-ef9e-4c11-8a21-441df5f47fbb admin admin] PATH_INFO: `/v3/projects/3e505bba7e6e430499d98093cb549eb8/users/8553cfd95cc148c494537e31a80f7616/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:18.966021 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be21469-ef9e-4c11-8a21-441df5f47fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e505bba7e6e430499d98093cb549eb8, user_id=8553cfd95cc148c494537e31a80f7616, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:18.969677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be21469-ef9e-4c11-8a21-441df5f47fbb admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:18.983640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1348/2689] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:18 2022] PUT /identity/v3/projects/3e505bba7e6e430499d98093cb549eb8/users/8553cfd95cc148c494537e31a80f7616/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:18.988664 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518afd9-fff4-4933-aad7-b208f29f35f3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:18.999728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518afd9-fff4-4933-aad7-b208f29f35f3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:19.022888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:19.063484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716cf6e-fd1b-4bcd-9d04-58082c8815c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:19.064015 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716cf6e-fd1b-4bcd-9d04-58082c8815c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.064297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716cf6e-fd1b-4bcd-9d04-58082c8815c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.064585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716cf6e-fd1b-4bcd-9d04-58082c8815c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.065318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716cf6e-fd1b-4bcd-9d04-58082c8815c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:19.066882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716cf6e-fd1b-4bcd-9d04-58082c8815c2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:19.078002 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1342/2690] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:19.082303 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be21469-ef9e-4c11-8a21-441df5f47fbb admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:19.087819 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:19.095071 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be21469-ef9e-4c11-8a21-441df5f47fbb admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:19.100984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:19.101669 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.101930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.102211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] PATH_INFO: `/v3/projects/3e505bba7e6e430499d98093cb549eb8/users/8553cfd95cc148c494537e31a80f7616/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.104335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e505bba7e6e430499d98093cb549eb8, user_id=8553cfd95cc148c494537e31a80f7616, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:19.107031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:19.117920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1349/2691] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:19 2022] PUT /identity/v3/projects/3e505bba7e6e430499d98093cb549eb8/users/8553cfd95cc148c494537e31a80f7616/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:19.129927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff3f3-c072-4920-9398-b807241e9844 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.130221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff3f3-c072-4920-9398-b807241e9844 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.130497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ff3f3-c072-4920-9398-b807241e9844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.160402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b95ff3f3-c072-4920-9398-b807241e9844 None None] MFA Rules not processed for user `8553cfd95cc148c494537e31a80f7616`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:19.161638 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b95ff3f3-c072-4920-9398-b807241e9844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:19.229547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1343/2692] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:19.241304 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:19.243054 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:19.304340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6d6ba-1c5a-43d2-8da9-8c3ad1d1e958 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:19.308285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8741e80-fc75-411d-987d-605dec534ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:19.308930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8741e80-fc75-411d-987d-605dec534ebb service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.309216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8741e80-fc75-411d-987d-605dec534ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.309731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8741e80-fc75-411d-987d-605dec534ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.320633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8741e80-fc75-411d-987d-605dec534ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:19.322025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8741e80-fc75-411d-987d-605dec534ebb service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:19.333039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1350/2693] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:19.361713 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ff3f3-c072-4920-9398-b807241e9844 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:19.425455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ff3f3-c072-4920-9398-b807241e9844 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:19.430085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:19.430714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.430991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.431294 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] PATH_INFO: `/v3/users/5c510ba75c114429a140d69f6e003130` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.432437 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c510ba75c114429a140d69f6e003130)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:19.434383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:19.486064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c510ba75c114429a140d69f6e003130'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.488650 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:19.501163 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c510ba75c114429a140d69f6e003130'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.501903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c510ba75c114429a140d69f6e003130'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.502531 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c510ba75c114429a140d69f6e003130'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.510129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c510ba75c114429a140d69f6e003130'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:19.522578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1344/2694] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:19 2022] DELETE /identity/v3/users/5c510ba75c114429a140d69f6e003130 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:19.527084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e7d4-5006-45ed-9ac5-c37821cf968a None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.527467 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e7d4-5006-45ed-9ac5-c37821cf968a None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.527751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e7d4-5006-45ed-9ac5-c37821cf968a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.550587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-e886e7d4-5006-45ed-9ac5-c37821cf968a None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:19.551965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e886e7d4-5006-45ed-9ac5-c37821cf968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:19.618561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1351/2695] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:19.629871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:19.631307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:19.690336 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c94db-d342-4b83-9953-53a5ce5481eb admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:19.693939 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2039-e847-4395-84d6-e6a9a7988f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:19.694534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2039-e847-4395-84d6-e6a9a7988f30 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:19.694807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2039-e847-4395-84d6-e6a9a7988f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:19.695073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2039-e847-4395-84d6-e6a9a7988f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:19.704636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f2039-e847-4395-84d6-e6a9a7988f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:19.705725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f2039-e847-4395-84d6-e6a9a7988f30 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:19.715980 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1345/2696] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:20.021818 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e7d4-5006-45ed-9ac5-c37821cf968a None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:20.023643 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e886e7d4-5006-45ed-9ac5-c37821cf968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:20.038770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e7d4-5006-45ed-9ac5-c37821cf968a None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.046235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.046886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.047161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.047438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] PATH_INFO: `/v3/projects/ede2fb72ba184682878c1fe2edb1ba56` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.048639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede2fb72ba184682878c1fe2edb1ba56)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.051548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:20.102840 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede2fb72ba184682878c1fe2edb1ba56', 'request_id': 'req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.111613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede2fb72ba184682878c1fe2edb1ba56', 'request_id': 'req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.117550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede2fb72ba184682878c1fe2edb1ba56', 'request_id': 'req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.131082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] The token cache is being invalidate because project ede2fb72ba184682878c1fe2edb1ba56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:20.131376 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.132023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.133552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1352/2697] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:20 2022] DELETE /identity/v3/projects/ede2fb72ba184682878c1fe2edb1ba56 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:20.137527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2039-e847-4395-84d6-e6a9a7988f30 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:20.148385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2039-e847-4395-84d6-e6a9a7988f30 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.209599 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.210229 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.210508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.210778 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] PATH_INFO: `/v3/users/8553cfd95cc148c494537e31a80f7616` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.211907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8553cfd95cc148c494537e31a80f7616)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.213876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:20.264235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8553cfd95cc148c494537e31a80f7616'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.279821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8553cfd95cc148c494537e31a80f7616'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.281121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8553cfd95cc148c494537e31a80f7616'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.281747 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8553cfd95cc148c494537e31a80f7616'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.285495 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:20.289071 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aa0bd1bf-6943-49f1-a631-2b8302964eaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8553cfd95cc148c494537e31a80f7616'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.304340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1346/2698] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:20 2022] DELETE /identity/v3/users/8553cfd95cc148c494537e31a80f7616 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:20.603080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:20.604672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:20.629301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71b00e-9cce-44fc-bf2d-edf7f95be79d admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.644624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:20.693878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.694417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.694721 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.694988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] PATH_INFO: `/v3/projects/3e505bba7e6e430499d98093cb549eb8` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.696115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e505bba7e6e430499d98093cb549eb8)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.698074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:20.747960 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e505bba7e6e430499d98093cb549eb8', 'request_id': 'req-b9c178a5-254d-4527-afd2-3ec1b42f337c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.754694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e505bba7e6e430499d98093cb549eb8', 'request_id': 'req-b9c178a5-254d-4527-afd2-3ec1b42f337c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.761435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e505bba7e6e430499d98093cb549eb8', 'request_id': 'req-b9c178a5-254d-4527-afd2-3ec1b42f337c'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.778425 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] The token cache is being invalidate because project 3e505bba7e6e430499d98093cb549eb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:20.778892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.780937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b9c178a5-254d-4527-afd2-3ec1b42f337c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:20.782916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1353/2699] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:20 2022] DELETE /identity/v3/projects/3e505bba7e6e430499d98093cb549eb8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:20.788079 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d384a5-4baf-451d-98dc-2c2f35daf013 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.788402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d384a5-4baf-451d-98dc-2c2f35daf013 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.788741 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d384a5-4baf-451d-98dc-2c2f35daf013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.810569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c4d384a5-4baf-451d-98dc-2c2f35daf013 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:20.812266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c4d384a5-4baf-451d-98dc-2c2f35daf013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:20.890668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1347/2700] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:20.898510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5be4da-0b3f-4e31-a51c-180259a7f362 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.898798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5be4da-0b3f-4e31-a51c-180259a7f362 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.899077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5be4da-0b3f-4e31-a51c-180259a7f362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.920936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-1f5be4da-0b3f-4e31-a51c-180259a7f362 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:20.922221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1f5be4da-0b3f-4e31-a51c-180259a7f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:20.931927 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1354/2701] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:20.936729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d384a5-4baf-451d-98dc-2c2f35daf013 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:20.946399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d384a5-4baf-451d-98dc-2c2f35daf013 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.950796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.952098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.952371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.952683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.953328 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:20.953632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.955216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:20.970435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1348/2702] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:20.974683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5be4da-0b3f-4e31-a51c-180259a7f362 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:20.984811 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5be4da-0b3f-4e31-a51c-180259a7f362 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:20.989810 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32558ee-449a-44aa-977d-fcc0d7141668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:20.990401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32558ee-449a-44aa-977d-fcc0d7141668 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:20.990681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32558ee-449a-44aa-977d-fcc0d7141668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:20.991016 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32558ee-449a-44aa-977d-fcc0d7141668 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:20.991668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32558ee-449a-44aa-977d-fcc0d7141668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:20.993611 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32558ee-449a-44aa-977d-fcc0d7141668 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.021872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1355/2703] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:20 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:21.026033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.038983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3751f5f-6711-4109-9747-a55e23613190 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.101225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d9bf8-54af-401b-8384-dc52ed6653db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.101857 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9bf8-54af-401b-8384-dc52ed6653db admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.102134 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9bf8-54af-401b-8384-dc52ed6653db admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.102408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9bf8-54af-401b-8384-dc52ed6653db admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.103034 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d9bf8-54af-401b-8384-dc52ed6653db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.105042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d9bf8-54af-401b-8384-dc52ed6653db admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.131281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1349/2704] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:21.135380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32558ee-449a-44aa-977d-fcc0d7141668 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.147536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32558ee-449a-44aa-977d-fcc0d7141668 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.152482 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9681489-fff6-43d3-abb6-0c388f7829b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.153116 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9681489-fff6-43d3-abb6-0c388f7829b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.153395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9681489-fff6-43d3-abb6-0c388f7829b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.153726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9681489-fff6-43d3-abb6-0c388f7829b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.154997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9681489-fff6-43d3-abb6-0c388f7829b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.157039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9681489-fff6-43d3-abb6-0c388f7829b0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.167333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1356/2705] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:21.172017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d9bf8-54af-401b-8384-dc52ed6653db admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.183154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d9bf8-54af-401b-8384-dc52ed6653db admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.187896 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c65c1-a333-4e4a-86bb-d17b2aa6cba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.188488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c65c1-a333-4e4a-86bb-d17b2aa6cba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.188791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c65c1-a333-4e4a-86bb-d17b2aa6cba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.189060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c65c1-a333-4e4a-86bb-d17b2aa6cba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.189726 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c65c1-a333-4e4a-86bb-d17b2aa6cba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.191300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c65c1-a333-4e4a-86bb-d17b2aa6cba3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.203448 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1350/2706] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:21.208524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9681489-fff6-43d3-abb6-0c388f7829b0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.219671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9681489-fff6-43d3-abb6-0c388f7829b0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.224515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febcdd7-7510-4d79-a76b-f0a4abe559ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.225202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febcdd7-7510-4d79-a76b-f0a4abe559ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.225485 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febcdd7-7510-4d79-a76b-f0a4abe559ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.225761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febcdd7-7510-4d79-a76b-f0a4abe559ce admin admin] PATH_INFO: `/v3/projects/6f32abd6656f4612a13163d3646ff77c/users/b080787d91bd4858bff101c95976424a/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.241476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febcdd7-7510-4d79-a76b-f0a4abe559ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f32abd6656f4612a13163d3646ff77c, user_id=b080787d91bd4858bff101c95976424a, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.244377 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febcdd7-7510-4d79-a76b-f0a4abe559ce admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.261025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1357/2707] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:21 2022] PUT /identity/v3/projects/6f32abd6656f4612a13163d3646ff77c/users/b080787d91bd4858bff101c95976424a/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:21.273381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a4987-d6f0-4da1-b9a7-22fd8255eb48 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.273818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a4987-d6f0-4da1-b9a7-22fd8255eb48 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.274092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a4987-d6f0-4da1-b9a7-22fd8255eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.304161 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-af8a4987-d6f0-4da1-b9a7-22fd8255eb48 None None] MFA Rules not processed for user `b080787d91bd4858bff101c95976424a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:21.305435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-af8a4987-d6f0-4da1-b9a7-22fd8255eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.349510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:21.374454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1351/2708] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:21.380109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febcdd7-7510-4d79-a76b-f0a4abe559ce admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.430188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:21.449814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febcdd7-7510-4d79-a76b-f0a4abe559ce admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.456613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeada62-b85e-45f2-a073-ab9646986b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.457287 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeada62-b85e-45f2-a073-ab9646986b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.457523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeada62-b85e-45f2-a073-ab9646986b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.457801 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeada62-b85e-45f2-a073-ab9646986b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.458462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeada62-b85e-45f2-a073-ab9646986b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.460414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeada62-b85e-45f2-a073-ab9646986b29 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.488825 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1358/2709] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:21.492836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a4987-d6f0-4da1-b9a7-22fd8255eb48 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.506241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a4987-d6f0-4da1-b9a7-22fd8255eb48 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.566769 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4453dd-fba8-4d19-93f8-f68f97dc4661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.568309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4453dd-fba8-4d19-93f8-f68f97dc4661 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.568639 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4453dd-fba8-4d19-93f8-f68f97dc4661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.569391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4453dd-fba8-4d19-93f8-f68f97dc4661 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.569914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4453dd-fba8-4d19-93f8-f68f97dc4661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.571642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4453dd-fba8-4d19-93f8-f68f97dc4661 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.595812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1352/2710] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:21.599652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeada62-b85e-45f2-a073-ab9646986b29 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.610259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeada62-b85e-45f2-a073-ab9646986b29 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.614947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b96544-a375-4db2-b319-d6427b11d77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.615547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b96544-a375-4db2-b319-d6427b11d77f admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.615830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b96544-a375-4db2-b319-d6427b11d77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.616094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b96544-a375-4db2-b319-d6427b11d77f admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.616761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b96544-a375-4db2-b319-d6427b11d77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.618339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b96544-a375-4db2-b319-d6427b11d77f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.629792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1359/2711] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:21.634499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4453dd-fba8-4d19-93f8-f68f97dc4661 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.645522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4453dd-fba8-4d19-93f8-f68f97dc4661 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.650138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4602b17-b166-4c5d-b9ec-05f9db2a086c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.650787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4602b17-b166-4c5d-b9ec-05f9db2a086c admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.651062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4602b17-b166-4c5d-b9ec-05f9db2a086c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.651332 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4602b17-b166-4c5d-b9ec-05f9db2a086c admin admin] PATH_INFO: `/v3/projects/640f5e265c894929a828ee65574f20af/users/5d1d715728d9444bbeae3cd5ef0c9cd8/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.666473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4602b17-b166-4c5d-b9ec-05f9db2a086c admin admin] RBAC: Authorizing `identity:create_grant(project_id=640f5e265c894929a828ee65574f20af, user_id=5d1d715728d9444bbeae3cd5ef0c9cd8, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.669141 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4602b17-b166-4c5d-b9ec-05f9db2a086c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.684440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1353/2712] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:21 2022] PUT /identity/v3/projects/640f5e265c894929a828ee65574f20af/users/5d1d715728d9444bbeae3cd5ef0c9cd8/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:21.689333 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b96544-a375-4db2-b319-d6427b11d77f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.699744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b96544-a375-4db2-b319-d6427b11d77f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.759567 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd2ce2-281e-4204-8ad9-19cbd07977e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.760215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2ce2-281e-4204-8ad9-19cbd07977e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.760530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2ce2-281e-4204-8ad9-19cbd07977e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.760864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd2ce2-281e-4204-8ad9-19cbd07977e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.761512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd2ce2-281e-4204-8ad9-19cbd07977e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.763195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd2ce2-281e-4204-8ad9-19cbd07977e0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.776138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1360/2713] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:21.781060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4602b17-b166-4c5d-b9ec-05f9db2a086c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.791296 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4602b17-b166-4c5d-b9ec-05f9db2a086c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:21.796391 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:21.797059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.797335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.797602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] PATH_INFO: `/v3/projects/640f5e265c894929a828ee65574f20af/users/5d1d715728d9444bbeae3cd5ef0c9cd8/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.799496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] RBAC: Authorizing `identity:create_grant(project_id=640f5e265c894929a828ee65574f20af, user_id=5d1d715728d9444bbeae3cd5ef0c9cd8, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:21.802123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:21.815565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1354/2714] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:21 2022] PUT /identity/v3/projects/640f5e265c894929a828ee65574f20af/users/5d1d715728d9444bbeae3cd5ef0c9cd8/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:21.826004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894574-ff89-4585-bc29-454a1652824e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:21.826327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894574-ff89-4585-bc29-454a1652824e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:21.826604 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83894574-ff89-4585-bc29-454a1652824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:21.856972 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-83894574-ff89-4585-bc29-454a1652824e None None] MFA Rules not processed for user `5d1d715728d9444bbeae3cd5ef0c9cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:21.859226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-83894574-ff89-4585-bc29-454a1652824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:21.932274 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1361/2715] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:21.946491 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:21.948228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:22.008145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ffffb-9476-4153-b39c-559e4f2d039b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.011692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917dfd9-e4b0-4741-910b-78bbfabe4414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.012284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917dfd9-e4b0-4741-910b-78bbfabe4414 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.012555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917dfd9-e4b0-4741-910b-78bbfabe4414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.012856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917dfd9-e4b0-4741-910b-78bbfabe4414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.021943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917dfd9-e4b0-4741-910b-78bbfabe4414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.023841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917dfd9-e4b0-4741-910b-78bbfabe4414 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.036691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1355/2716] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:22.125898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83894574-ff89-4585-bc29-454a1652824e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:22.127312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-83894574-ff89-4585-bc29-454a1652824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:22.151724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83894574-ff89-4585-bc29-454a1652824e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:22.156337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5086d-f5ca-479f-96dc-2882c1349557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:22.156986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5086d-f5ca-479f-96dc-2882c1349557 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:22.157269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5086d-f5ca-479f-96dc-2882c1349557 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:22.157541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5086d-f5ca-479f-96dc-2882c1349557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:22.194627 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:22.222019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5086d-f5ca-479f-96dc-2882c1349557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:22.223534 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5086d-f5ca-479f-96dc-2882c1349557 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:22.233620 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1362/2717] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:27:27.046830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917dfd9-e4b0-4741-910b-78bbfabe4414 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:27.093786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:27.115366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917dfd9-e4b0-4741-910b-78bbfabe4414 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:27.120424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:27.121143 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:27.121434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:27.121684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] PATH_INFO: `/v3/users/b080787d91bd4858bff101c95976424a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:27.123230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b080787d91bd4858bff101c95976424a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:27.125310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:27.174809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b080787d91bd4858bff101c95976424a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.187201 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b080787d91bd4858bff101c95976424a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.187896 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b080787d91bd4858bff101c95976424a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.188500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b080787d91bd4858bff101c95976424a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.197461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b080787d91bd4858bff101c95976424a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.211744 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1356/2718] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:27 2022] DELETE /identity/v3/users/b080787d91bd4858bff101c95976424a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:27.217216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b0ff-ae8a-4fcf-b49f-7c378a12d5b5 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:27.218128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b0ff-ae8a-4fcf-b49f-7c378a12d5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:27.218128 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828b0ff-ae8a-4fcf-b49f-7c378a12d5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:27.236432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-c828b0ff-ae8a-4fcf-b49f-7c378a12d5b5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:27.237734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c828b0ff-ae8a-4fcf-b49f-7c378a12d5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:27.298314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1363/2719] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:27.308027 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:27.309436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:27.369917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dae4c3-3cb8-47ae-9adb-6c0b8efaa5ec admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:27.374117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476faf6-f682-486c-abcb-148c9fa6b291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:27.374720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476faf6-f682-486c-abcb-148c9fa6b291 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:27.375001 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476faf6-f682-486c-abcb-148c9fa6b291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:27.375272 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476faf6-f682-486c-abcb-148c9fa6b291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:27.384610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476faf6-f682-486c-abcb-148c9fa6b291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:27.386043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476faf6-f682-486c-abcb-148c9fa6b291 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:27.397048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1357/2720] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:27.677962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828b0ff-ae8a-4fcf-b49f-7c378a12d5b5 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:27.679119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c828b0ff-ae8a-4fcf-b49f-7c378a12d5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:27.692187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828b0ff-ae8a-4fcf-b49f-7c378a12d5b5 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:27.696985 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:27.697621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:27.697902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:27.698178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] PATH_INFO: `/v3/projects/6f32abd6656f4612a13163d3646ff77c` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:27.699314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f32abd6656f4612a13163d3646ff77c)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:27.702044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:27.748761 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f32abd6656f4612a13163d3646ff77c', 'request_id': 'req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.755034 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f32abd6656f4612a13163d3646ff77c', 'request_id': 'req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.761592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f32abd6656f4612a13163d3646ff77c', 'request_id': 'req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.773860 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] The token cache is being invalidate because project 6f32abd6656f4612a13163d3646ff77c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:27.774147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.774775 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.776283 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1364/2721] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:27 2022] DELETE /identity/v3/projects/6f32abd6656f4612a13163d3646ff77c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:27.781209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476faf6-f682-486c-abcb-148c9fa6b291 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:27.791004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476faf6-f682-486c-abcb-148c9fa6b291 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:27.849964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:27.850499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:27.850752 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:27.851025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] PATH_INFO: `/v3/users/5d1d715728d9444bbeae3cd5ef0c9cd8` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:27.852284 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1d715728d9444bbeae3cd5ef0c9cd8)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:27.854255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:27.855746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:27.903738 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1d715728d9444bbeae3cd5ef0c9cd8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.916771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1d715728d9444bbeae3cd5ef0c9cd8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.917759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1d715728d9444bbeae3cd5ef0c9cd8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.918396 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1d715728d9444bbeae3cd5ef0c9cd8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.926921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6d58293b-11a1-42bf-af00-17508211f9d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1d715728d9444bbeae3cd5ef0c9cd8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:27.940560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1358/2722] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:27 2022] DELETE /identity/v3/users/5d1d715728d9444bbeae3cd5ef0c9cd8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:28.227814 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.229639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:28.244543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4f997-50bb-4ec2-a7bb-3e2acc32b5f7 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.306939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.307575 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.307873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.308127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] PATH_INFO: `/v3/projects/640f5e265c894929a828ee65574f20af` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.309269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=640f5e265c894929a828ee65574f20af)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.311829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.313459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:28.362206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640f5e265c894929a828ee65574f20af', 'request_id': 'req-5f8fcdae-cf45-4d46-ab05-d627ee64a043'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.368300 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640f5e265c894929a828ee65574f20af', 'request_id': 'req-5f8fcdae-cf45-4d46-ab05-d627ee64a043'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.374031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640f5e265c894929a828ee65574f20af', 'request_id': 'req-5f8fcdae-cf45-4d46-ab05-d627ee64a043'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.387049 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] The token cache is being invalidate because project 640f5e265c894929a828ee65574f20af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:28.387338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.387955 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-5f8fcdae-cf45-4d46-ab05-d627ee64a043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:28.389590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1365/2723] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:28 2022] DELETE /identity/v3/projects/640f5e265c894929a828ee65574f20af => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:28.395249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc02fca-5c29-44c1-be5f-5d80855e7154 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.395733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc02fca-5c29-44c1-be5f-5d80855e7154 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.395854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc02fca-5c29-44c1-be5f-5d80855e7154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.416493 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3dc02fca-5c29-44c1-be5f-5d80855e7154 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:28.417733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3dc02fca-5c29-44c1-be5f-5d80855e7154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:28.482867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1359/2724] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:28.490760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80636593-04d2-487d-b142-0a4c9be76e5b None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.491132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80636593-04d2-487d-b142-0a4c9be76e5b None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.491478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80636593-04d2-487d-b142-0a4c9be76e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.512294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-80636593-04d2-487d-b142-0a4c9be76e5b None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:28.514185 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-80636593-04d2-487d-b142-0a4c9be76e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:28.522046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1366/2725] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:28.526329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc02fca-5c29-44c1-be5f-5d80855e7154 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.536924 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc02fca-5c29-44c1-be5f-5d80855e7154 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.541406 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.541998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.542277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.542551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.543191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:28.543486 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.545178 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.558326 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1360/2726] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:28.562863 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80636593-04d2-487d-b142-0a4c9be76e5b None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.572309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80636593-04d2-487d-b142-0a4c9be76e5b None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.577771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54281831-0da0-4386-a3a9-af96da7606ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.578361 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54281831-0da0-4386-a3a9-af96da7606ae admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.578640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54281831-0da0-4386-a3a9-af96da7606ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.578911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54281831-0da0-4386-a3a9-af96da7606ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.579549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54281831-0da0-4386-a3a9-af96da7606ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.581495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54281831-0da0-4386-a3a9-af96da7606ae admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.618889 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1367/2727] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:28.623342 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.633364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42d347-fb37-493a-aab4-b1094fb8dfd5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.689440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3ffc7-b166-4a54-aa49-73fda4f47fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.690044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3ffc7-b166-4a54-aa49-73fda4f47fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.690320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3ffc7-b166-4a54-aa49-73fda4f47fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.690586 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3ffc7-b166-4a54-aa49-73fda4f47fb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.691225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3ffc7-b166-4a54-aa49-73fda4f47fb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.693183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3ffc7-b166-4a54-aa49-73fda4f47fb8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.706151 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:28.724689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1361/2728] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:28.728213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54281831-0da0-4386-a3a9-af96da7606ae admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.739102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54281831-0da0-4386-a3a9-af96da7606ae admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.743997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6baac2-c7ea-4a91-8e2a-fd60ead73483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.744772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6baac2-c7ea-4a91-8e2a-fd60ead73483 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.745150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6baac2-c7ea-4a91-8e2a-fd60ead73483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.745512 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6baac2-c7ea-4a91-8e2a-fd60ead73483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.746253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6baac2-c7ea-4a91-8e2a-fd60ead73483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.748374 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6baac2-c7ea-4a91-8e2a-fd60ead73483 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.759539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1368/2729] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:28.764885 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3ffc7-b166-4a54-aa49-73fda4f47fb8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.775106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3ffc7-b166-4a54-aa49-73fda4f47fb8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.780988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a682c19-34b0-440a-a0d5-7d2eb9a073f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.781605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a682c19-34b0-440a-a0d5-7d2eb9a073f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.781887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a682c19-34b0-440a-a0d5-7d2eb9a073f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.782160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a682c19-34b0-440a-a0d5-7d2eb9a073f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.782841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a682c19-34b0-440a-a0d5-7d2eb9a073f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.784496 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a682c19-34b0-440a-a0d5-7d2eb9a073f1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.797255 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1362/2730] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:28.802107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6baac2-c7ea-4a91-8e2a-fd60ead73483 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:28.815226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6baac2-c7ea-4a91-8e2a-fd60ead73483 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:28.819988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:28.820691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.820968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.821371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] PATH_INFO: `/v3/projects/d80d031d5b9f4e0f89621f7e9329a15f/users/78dd7adb0e864fb5b27626df50516a81/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.837765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80d031d5b9f4e0f89621f7e9329a15f, user_id=78dd7adb0e864fb5b27626df50516a81, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:28.840460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:28.853631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1369/2731] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:28 2022] PUT /identity/v3/projects/d80d031d5b9f4e0f89621f7e9329a15f/users/78dd7adb0e864fb5b27626df50516a81/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:28.865518 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468b83c-61db-4de3-89f9-a53f231691a0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:28.865803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468b83c-61db-4de3-89f9-a53f231691a0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:28.866049 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468b83c-61db-4de3-89f9-a53f231691a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:28.899247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1468b83c-61db-4de3-89f9-a53f231691a0 None None] MFA Rules not processed for user `78dd7adb0e864fb5b27626df50516a81`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:28.900533 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1468b83c-61db-4de3-89f9-a53f231691a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:28.988551 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1363/2732] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:29.000654 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:29.002403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:29.067703 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1790e-859f-49e0-90da-a4b61b36e5de admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:29.071368 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c63cb-3bdd-40a5-b449-26df5494723e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:29.071978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c63cb-3bdd-40a5-b449-26df5494723e service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:29.072254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c63cb-3bdd-40a5-b449-26df5494723e service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:29.072523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c63cb-3bdd-40a5-b449-26df5494723e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:29.082073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c63cb-3bdd-40a5-b449-26df5494723e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:29.083521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c63cb-3bdd-40a5-b449-26df5494723e service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:29.092422 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1370/2733] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:31.675609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468b83c-61db-4de3-89f9-a53f231691a0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:31.739211 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468b83c-61db-4de3-89f9-a53f231691a0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:31.743731 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:31.744350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:31.744663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:31.744935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] PATH_INFO: `/v3/users/78dd7adb0e864fb5b27626df50516a81` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:31.746047 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] RBAC: Authorizing `identity:delete_user(user_id=78dd7adb0e864fb5b27626df50516a81)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:31.748408 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:31.798813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78dd7adb0e864fb5b27626df50516a81'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:31.810418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78dd7adb0e864fb5b27626df50516a81'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:31.811104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78dd7adb0e864fb5b27626df50516a81'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:31.811709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78dd7adb0e864fb5b27626df50516a81'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:31.819428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78dd7adb0e864fb5b27626df50516a81'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:31.824003 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:31.834384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1364/2734] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:31 2022] DELETE /identity/v3/users/78dd7adb0e864fb5b27626df50516a81 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:31.838680 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701550a-66c0-4aee-8bda-88288f6a91cf None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:31.839151 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701550a-66c0-4aee-8bda-88288f6a91cf None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:31.839376 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701550a-66c0-4aee-8bda-88288f6a91cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:31.859733 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-8701550a-66c0-4aee-8bda-88288f6a91cf None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:31.860977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8701550a-66c0-4aee-8bda-88288f6a91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:31.874529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:31.926451 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1371/2735] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:31.936829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:31.938241 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:32.003465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875637ff-e5eb-4a4b-bef0-ff5cdcb9667e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:32.011513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6bc35-45cd-422a-9972-0309daef428d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:32.012194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6bc35-45cd-422a-9972-0309daef428d service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.012474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6bc35-45cd-422a-9972-0309daef428d service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.012772 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6bc35-45cd-422a-9972-0309daef428d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.023694 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6bc35-45cd-422a-9972-0309daef428d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:32.025401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6bc35-45cd-422a-9972-0309daef428d service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:32.035954 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1365/2736] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:32.334905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701550a-66c0-4aee-8bda-88288f6a91cf None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:32.337122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-8701550a-66c0-4aee-8bda-88288f6a91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:32.352453 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701550a-66c0-4aee-8bda-88288f6a91cf None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:32.359243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:32.359243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.359243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.359243 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] PATH_INFO: `/v3/projects/d80d031d5b9f4e0f89621f7e9329a15f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.366578 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d80d031d5b9f4e0f89621f7e9329a15f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:32.367194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:32.415981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd80d031d5b9f4e0f89621f7e9329a15f', 'request_id': 'req-60866e4b-0487-4572-9ca8-3cb644fb7708'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:32.423130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd80d031d5b9f4e0f89621f7e9329a15f', 'request_id': 'req-60866e4b-0487-4572-9ca8-3cb644fb7708'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:32.429077 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd80d031d5b9f4e0f89621f7e9329a15f', 'request_id': 'req-60866e4b-0487-4572-9ca8-3cb644fb7708'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:32.443219 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] The token cache is being invalidate because project d80d031d5b9f4e0f89621f7e9329a15f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:32.443423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:32.444094 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-60866e4b-0487-4572-9ca8-3cb644fb7708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:32.445630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1372/2737] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:32 2022] DELETE /identity/v3/projects/d80d031d5b9f4e0f89621f7e9329a15f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:32.451046 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c141acb-8b99-406b-b1a9-5dd15e1b625e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.451387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c141acb-8b99-406b-b1a9-5dd15e1b625e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.451863 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c141acb-8b99-406b-b1a9-5dd15e1b625e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.473622 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3c141acb-8b99-406b-b1a9-5dd15e1b625e None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:32.474839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3c141acb-8b99-406b-b1a9-5dd15e1b625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:32.538379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1366/2738] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:32.547010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19a6ea-bbfd-4572-b0ca-7c0b31df17ee None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.547613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19a6ea-bbfd-4572-b0ca-7c0b31df17ee None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.547999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19a6ea-bbfd-4572-b0ca-7c0b31df17ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.569276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fb19a6ea-bbfd-4572-b0ca-7c0b31df17ee None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:32.570522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fb19a6ea-bbfd-4572-b0ca-7c0b31df17ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:32.579263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1373/2739] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:32.583602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c141acb-8b99-406b-b1a9-5dd15e1b625e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:32.594234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c141acb-8b99-406b-b1a9-5dd15e1b625e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:32.598734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:32.599319 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.599756 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.600012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.600689 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:32.600988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:32.602590 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:32.615191 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1367/2740] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:32.619837 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19a6ea-bbfd-4572-b0ca-7c0b31df17ee None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:32.629898 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19a6ea-bbfd-4572-b0ca-7c0b31df17ee None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:32.634795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443d0b6-1dc6-4b72-9fa6-fe1dca058dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:32.635395 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d0b6-1dc6-4b72-9fa6-fe1dca058dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.635673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d0b6-1dc6-4b72-9fa6-fe1dca058dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.635943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d0b6-1dc6-4b72-9fa6-fe1dca058dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.636609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443d0b6-1dc6-4b72-9fa6-fe1dca058dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:32.639406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443d0b6-1dc6-4b72-9fa6-fe1dca058dd2 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:32.667661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1374/2741] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:32 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:32.671933 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:32.683133 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4235-c825-416d-b377-72d42a63948f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:32.746348 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46841-e78c-493e-ac70-4a078d2dfaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:32.746960 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46841-e78c-493e-ac70-4a078d2dfaef admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.747266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46841-e78c-493e-ac70-4a078d2dfaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.748144 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f46841-e78c-493e-ac70-4a078d2dfaef admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.748890 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f46841-e78c-493e-ac70-4a078d2dfaef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:32.750864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f46841-e78c-493e-ac70-4a078d2dfaef admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:32.774429 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1368/2742] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:32 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:32.780142 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443d0b6-1dc6-4b72-9fa6-fe1dca058dd2 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:32.791095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443d0b6-1dc6-4b72-9fa6-fe1dca058dd2 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:32.796385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67143884-b399-4f7e-9535-baa661ff7f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:32.797165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67143884-b399-4f7e-9535-baa661ff7f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.797470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67143884-b399-4f7e-9535-baa661ff7f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.797747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67143884-b399-4f7e-9535-baa661ff7f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.798398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67143884-b399-4f7e-9535-baa661ff7f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:32.800103 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67143884-b399-4f7e-9535-baa661ff7f9e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:32.811138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1375/2743] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:32.815059 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46841-e78c-493e-ac70-4a078d2dfaef admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:32.825158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f46841-e78c-493e-ac70-4a078d2dfaef admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:32.830262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472df246-303d-4249-aba8-a912a863431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:32.830846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472df246-303d-4249-aba8-a912a863431f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.831121 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472df246-303d-4249-aba8-a912a863431f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.831499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472df246-303d-4249-aba8-a912a863431f admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.832097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472df246-303d-4249-aba8-a912a863431f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:32.833703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472df246-303d-4249-aba8-a912a863431f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:32.845314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1369/2744] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:32.849856 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67143884-b399-4f7e-9535-baa661ff7f9e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:32.860549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67143884-b399-4f7e-9535-baa661ff7f9e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:32.865266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:32.865855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.866132 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.866406 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] PATH_INFO: `/v3/projects/15872120c9864a7a84c7705fde43e71b/users/438ab0283a314afd8ba9da4f313c6eb4/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.882393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15872120c9864a7a84c7705fde43e71b, user_id=438ab0283a314afd8ba9da4f313c6eb4, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:32.885188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:32.899286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1376/2745] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:32 2022] PUT /identity/v3/projects/15872120c9864a7a84c7705fde43e71b/users/438ab0283a314afd8ba9da4f313c6eb4/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:32.911479 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff551e9-7323-467f-9838-3443df82605b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:32.911948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff551e9-7323-467f-9838-3443df82605b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:32.912289 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff551e9-7323-467f-9838-3443df82605b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:32.920405 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:32.944797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-bff551e9-7323-467f-9838-3443df82605b None None] MFA Rules not processed for user `438ab0283a314afd8ba9da4f313c6eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:32.946023 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bff551e9-7323-467f-9838-3443df82605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:33.010892 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1370/2746] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:33.022058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:33.023842 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:33.071609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:33.090736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f6b82-bc99-492c-a27b-b80e1ad8c360 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:33.095233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de2284-f87e-4397-93c7-ed90062c6c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:33.095994 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de2284-f87e-4397-93c7-ed90062c6c74 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:33.096301 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de2284-f87e-4397-93c7-ed90062c6c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:33.096592 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de2284-f87e-4397-93c7-ed90062c6c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:33.105626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de2284-f87e-4397-93c7-ed90062c6c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:33.107660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de2284-f87e-4397-93c7-ed90062c6c74 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:33.117388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1377/2747] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:33.881497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff551e9-7323-467f-9838-3443df82605b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:33.883146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bff551e9-7323-467f-9838-3443df82605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:33.958583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff551e9-7323-467f-9838-3443df82605b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:33.962753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:33.963361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:33.963634 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:33.963903 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:33.965724 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:34.050385 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:34.051728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:34.064445 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1371/2748] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:35.277911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de2284-f87e-4397-93c7-ed90062c6c74 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:35.288450 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de2284-f87e-4397-93c7-ed90062c6c74 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:35.292037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1579-38fc-4009-9651-4cd87f07579b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:35.292681 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1579-38fc-4009-9651-4cd87f07579b service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:35.292980 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1579-38fc-4009-9651-4cd87f07579b service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:35.293257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1579-38fc-4009-9651-4cd87f07579b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:35.295037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1add1579-38fc-4009-9651-4cd87f07579b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:35.309140 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add1579-38fc-4009-9651-4cd87f07579b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:35.310522 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add1579-38fc-4009-9651-4cd87f07579b service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:35.321661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1378/2749] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:35.686907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:35.688413 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:35.728309 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:35.758530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d5a04-b7cf-4d38-af79-cb0638c87d8c service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:35.763922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:35.765031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:35.765031 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:35.765337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] PATH_INFO: `/v3/users/26a267ca9b6f443ebdcd39ea1a5ed537` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:35.766354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a267ca9b6f443ebdcd39ea1a5ed537)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:35.768286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:35.814732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a267ca9b6f443ebdcd39ea1a5ed537'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:35.827206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a267ca9b6f443ebdcd39ea1a5ed537'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:35.828181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a267ca9b6f443ebdcd39ea1a5ed537'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:35.828819 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a267ca9b6f443ebdcd39ea1a5ed537'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:35.836156 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a267ca9b6f443ebdcd39ea1a5ed537'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:35.848281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1372/2750] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:35 2022] DELETE /identity/v3/users/26a267ca9b6f443ebdcd39ea1a5ed537 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:35.852759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11812c-3dea-4087-b1a0-2241be979d84 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:35.853053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11812c-3dea-4087-b1a0-2241be979d84 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:35.853330 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11812c-3dea-4087-b1a0-2241be979d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:35.873215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-7e11812c-3dea-4087-b1a0-2241be979d84 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:35.874429 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7e11812c-3dea-4087-b1a0-2241be979d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:35.935672 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1379/2751] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:35.945102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:35.946755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:36.006931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5146-0102-4c89-846c-0f20c20ea7b1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:36.010692 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bcb9d-7d63-45c4-9cc6-06f6b3c60f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:36.011291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bcb9d-7d63-45c4-9cc6-06f6b3c60f98 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.011578 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bcb9d-7d63-45c4-9cc6-06f6b3c60f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.011869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bcb9d-7d63-45c4-9cc6-06f6b3c60f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.023234 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bcb9d-7d63-45c4-9cc6-06f6b3c60f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:36.025067 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bcb9d-7d63-45c4-9cc6-06f6b3c60f98 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:36.037193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1373/2752] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:36.340254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11812c-3dea-4087-b1a0-2241be979d84 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:36.342431 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7e11812c-3dea-4087-b1a0-2241be979d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:36.356082 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11812c-3dea-4087-b1a0-2241be979d84 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:36.362389 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:36.363149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.363298 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.363571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] PATH_INFO: `/v3/projects/614275b03a454a548ce65fe8264b49a2` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.364751 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614275b03a454a548ce65fe8264b49a2)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:36.366675 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:36.414259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614275b03a454a548ce65fe8264b49a2', 'request_id': 'req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:36.421328 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614275b03a454a548ce65fe8264b49a2', 'request_id': 'req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:36.427172 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614275b03a454a548ce65fe8264b49a2', 'request_id': 'req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:36.439926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] The token cache is being invalidate because project 614275b03a454a548ce65fe8264b49a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:36.440386 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:36.440892 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b0dd3c37-0aac-4eb4-9e63-7fd28dee5d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:36.442396 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1380/2753] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:36 2022] DELETE /identity/v3/projects/614275b03a454a548ce65fe8264b49a2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:36.447358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544324f-13a3-4150-8b83-36e0f0cf8aad None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.447651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544324f-13a3-4150-8b83-36e0f0cf8aad None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.447936 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544324f-13a3-4150-8b83-36e0f0cf8aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.470989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9544324f-13a3-4150-8b83-36e0f0cf8aad None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:36.472216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9544324f-13a3-4150-8b83-36e0f0cf8aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:36.525243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:36.539280 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1374/2754] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:36.547870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837525d9-9940-47b5-9792-d20733224d13 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.548470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837525d9-9940-47b5-9792-d20733224d13 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.548782 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837525d9-9940-47b5-9792-d20733224d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.570030 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-837525d9-9940-47b5-9792-d20733224d13 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:36.571263 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-837525d9-9940-47b5-9792-d20733224d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:36.579559 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1381/2755] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:27:36.583932 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544324f-13a3-4150-8b83-36e0f0cf8aad None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:36.594190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544324f-13a3-4150-8b83-36e0f0cf8aad None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:36.598944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:36.599556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.599967 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.600243 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.600935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:36.601238 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:36.602841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:36.617230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1375/2756] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:36.621704 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837525d9-9940-47b5-9792-d20733224d13 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:36.632058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837525d9-9940-47b5-9792-d20733224d13 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:36.637002 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e8cb9-1de9-4485-820a-473257ae2758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:36.637597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e8cb9-1de9-4485-820a-473257ae2758 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.637903 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e8cb9-1de9-4485-820a-473257ae2758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.638182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e8cb9-1de9-4485-820a-473257ae2758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.639018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e8cb9-1de9-4485-820a-473257ae2758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:36.640965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e8cb9-1de9-4485-820a-473257ae2758 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:36.671122 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1382/2757] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:36 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:36.675444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:36.685806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dd1c0-f293-47db-a23f-652bc4fd97ef admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:36.744450 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61674bd-4819-47ee-ac98-2f0504c7680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:36.745155 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61674bd-4819-47ee-ac98-2f0504c7680b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.745436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61674bd-4819-47ee-ac98-2f0504c7680b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.745706 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61674bd-4819-47ee-ac98-2f0504c7680b admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.746436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61674bd-4819-47ee-ac98-2f0504c7680b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:36.748424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61674bd-4819-47ee-ac98-2f0504c7680b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:36.771099 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1376/2758] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:36 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:36.775310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e8cb9-1de9-4485-820a-473257ae2758 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:36.786941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e8cb9-1de9-4485-820a-473257ae2758 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:36.791673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b18796-9e26-4a58-a355-f750ff570414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:36.792269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b18796-9e26-4a58-a355-f750ff570414 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.792543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b18796-9e26-4a58-a355-f750ff570414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.792891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b18796-9e26-4a58-a355-f750ff570414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.793532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b18796-9e26-4a58-a355-f750ff570414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:36.795160 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b18796-9e26-4a58-a355-f750ff570414 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:36.805063 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1383/2759] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:36.809106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61674bd-4819-47ee-ac98-2f0504c7680b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:36.819754 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61674bd-4819-47ee-ac98-2f0504c7680b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:36.824418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d70e8b-25a7-4a6f-9fcb-595be1f7e48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:36.825044 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70e8b-25a7-4a6f-9fcb-595be1f7e48f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.825852 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70e8b-25a7-4a6f-9fcb-595be1f7e48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.826703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d70e8b-25a7-4a6f-9fcb-595be1f7e48f admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.827359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d70e8b-25a7-4a6f-9fcb-595be1f7e48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:36.828990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d70e8b-25a7-4a6f-9fcb-595be1f7e48f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:36.839364 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1377/2760] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:36.844398 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b18796-9e26-4a58-a355-f750ff570414 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:36.854650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b18796-9e26-4a58-a355-f750ff570414 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:36.859673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:36.860324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.860652 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.861043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] PATH_INFO: `/v3/projects/a6dfd4d42e874602b3432b46894f2aaa/users/e8789ed99cb74d9183fe770c0b72cae6/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.872265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:36.877490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dfd4d42e874602b3432b46894f2aaa, user_id=e8789ed99cb74d9183fe770c0b72cae6, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:36.880716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:36.894853 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1384/2761] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:36 2022] PUT /identity/v3/projects/a6dfd4d42e874602b3432b46894f2aaa/users/e8789ed99cb74d9183fe770c0b72cae6/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:36.907180 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a548df-d1f5-4739-9143-d81f9d02fe2b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:36.907473 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a548df-d1f5-4739-9143-d81f9d02fe2b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:36.907751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a548df-d1f5-4739-9143-d81f9d02fe2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:36.935447 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b4a548df-d1f5-4739-9143-d81f9d02fe2b None None] MFA Rules not processed for user `e8789ed99cb74d9183fe770c0b72cae6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:36.936712 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b4a548df-d1f5-4739-9143-d81f9d02fe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:36.999915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1378/2762] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:37.010636 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:37.012322 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:37.070383 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ca7f-23e1-4373-a2be-463a7d2bf734 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:37.074038 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770eaf2-711c-449c-a1bd-dd3ae8b85735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:37.074640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770eaf2-711c-449c-a1bd-dd3ae8b85735 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:37.074911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770eaf2-711c-449c-a1bd-dd3ae8b85735 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:37.075180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770eaf2-711c-449c-a1bd-dd3ae8b85735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:37.084259 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770eaf2-711c-449c-a1bd-dd3ae8b85735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:37.085735 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770eaf2-711c-449c-a1bd-dd3ae8b85735 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:37.095245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1385/2763] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:46.996813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:47.090393 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:49.198110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a548df-d1f5-4739-9143-d81f9d02fe2b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:49.199539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b4a548df-d1f5-4739-9143-d81f9d02fe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:49.257333 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a548df-d1f5-4739-9143-d81f9d02fe2b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:49.260913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbe2d3-2f07-4d36-9ab8-f97389510a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:49.261510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbe2d3-2f07-4d36-9ab8-f97389510a07 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:49.261787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbe2d3-2f07-4d36-9ab8-f97389510a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:49.262268 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbe2d3-2f07-4d36-9ab8-f97389510a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:49.264931 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-85bbe2d3-2f07-4d36-9ab8-f97389510a07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:49.278359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770eaf2-711c-449c-a1bd-dd3ae8b85735 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:49.333477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbe2d3-2f07-4d36-9ab8-f97389510a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:49.334928 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbe2d3-2f07-4d36-9ab8-f97389510a07 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:49.347543 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1379/2764] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:49.349009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770eaf2-711c-449c-a1bd-dd3ae8b85735 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:49.353623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:49.354341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:49.354631 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:49.354978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] PATH_INFO: `/v3/users/e8789ed99cb74d9183fe770c0b72cae6` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:49.357083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8789ed99cb74d9183fe770c0b72cae6)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:49.359019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:49.420857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8789ed99cb74d9183fe770c0b72cae6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.428472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8789ed99cb74d9183fe770c0b72cae6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.429234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8789ed99cb74d9183fe770c0b72cae6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.442350 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8789ed99cb74d9183fe770c0b72cae6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.445407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8789ed99cb74d9183fe770c0b72cae6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:49.459941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1386/2765] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:49 2022] DELETE /identity/v3/users/e8789ed99cb74d9183fe770c0b72cae6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:49.464600 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8c63f-5813-449d-888d-36093d856ad9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:49.464894 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8c63f-5813-449d-888d-36093d856ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:49.465171 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8c63f-5813-449d-888d-36093d856ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:49.485388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-37a8c63f-5813-449d-888d-36093d856ad9 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:49.486610 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-37a8c63f-5813-449d-888d-36093d856ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:49.548695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1380/2766] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:49.558345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:49.559997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:49.624245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba8913-2f41-496a-ae4b-b9998c608972 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:49.629131 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:49.629799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:49.630080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:49.630403 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:49.639737 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:49.641314 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:49.651205 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1387/2767] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:49.933559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8c63f-5813-449d-888d-36093d856ad9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:49.935108 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-37a8c63f-5813-449d-888d-36093d856ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:49.948048 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8c63f-5813-449d-888d-36093d856ad9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:49.954382 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:49.955280 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:49.955509 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:49.955842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] PATH_INFO: `/v3/projects/a6dfd4d42e874602b3432b46894f2aaa` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:49.957253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dfd4d42e874602b3432b46894f2aaa)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:49.959791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:50.009379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:50.012847 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dfd4d42e874602b3432b46894f2aaa', 'request_id': 'req-e3444cb9-862e-455a-b45c-cba6e2f179fc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.021768 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dfd4d42e874602b3432b46894f2aaa', 'request_id': 'req-e3444cb9-862e-455a-b45c-cba6e2f179fc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.028100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dfd4d42e874602b3432b46894f2aaa', 'request_id': 'req-e3444cb9-862e-455a-b45c-cba6e2f179fc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.040515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] The token cache is being invalidate because project a6dfd4d42e874602b3432b46894f2aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:50.040843 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.041497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:50.043078 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1381/2768] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:49 2022] DELETE /identity/v3/projects/a6dfd4d42e874602b3432b46894f2aaa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:52.639086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:52.640660 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:52.654141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3635d-a53c-454b-848d-c7f42e4f7faf service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:52.673946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:52.714224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:52.714844 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:52.715127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:52.715404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] PATH_INFO: `/v3/users/25c3f0a2e1fd4067b1804ef42faa46d5` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:52.717868 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:52.719573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:52.730574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c3f0a2e1fd4067b1804ef42faa46d5)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:52.732195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3444cb9-862e-455a-b45c-cba6e2f179fc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:52.732609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:52.737340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:52.737959 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:52.738237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:52.738559 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] PATH_INFO: `/v3/users/438ab0283a314afd8ba9da4f313c6eb4` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:52.739799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=438ab0283a314afd8ba9da4f313c6eb4)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:52.741743 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:52.786498 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c3f0a2e1fd4067b1804ef42faa46d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.792021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438ab0283a314afd8ba9da4f313c6eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.793774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c3f0a2e1fd4067b1804ef42faa46d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.794547 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c3f0a2e1fd4067b1804ef42faa46d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.805556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438ab0283a314afd8ba9da4f313c6eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.806372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438ab0283a314afd8ba9da4f313c6eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.806998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438ab0283a314afd8ba9da4f313c6eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.807266 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c3f0a2e1fd4067b1804ef42faa46d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.807902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-d0f3e899-c5c5-4e6b-b730-cc761d479402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c3f0a2e1fd4067b1804ef42faa46d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.816725 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438ab0283a314afd8ba9da4f313c6eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:52.825258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1388/2769] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:52 2022] DELETE /identity/v3/users/25c3f0a2e1fd4067b1804ef42faa46d5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:52.829912 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1382/2770] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:52 2022] DELETE /identity/v3/users/438ab0283a314afd8ba9da4f313c6eb4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:52.834124 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340319ad-c3f5-4330-884b-381691f8f8bc None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:52.834410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340319ad-c3f5-4330-884b-381691f8f8bc None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:52.834682 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340319ad-c3f5-4330-884b-381691f8f8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:52.855107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-340319ad-c3f5-4330-884b-381691f8f8bc None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:52.856335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-340319ad-c3f5-4330-884b-381691f8f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:52.919323 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1389/2771] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:52.930173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:52.931719 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:52.992541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b610985-09d1-4119-ae3a-cd2771fe52cd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:52.996351 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7c3fd-98f1-4f8a-a981-7d3994b531cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:52.997005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c3fd-98f1-4f8a-a981-7d3994b531cc service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:52.997286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c3fd-98f1-4f8a-a981-7d3994b531cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:52.997565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c3fd-98f1-4f8a-a981-7d3994b531cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.008233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7c3fd-98f1-4f8a-a981-7d3994b531cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.009733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7c3fd-98f1-4f8a-a981-7d3994b531cc service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.014929 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340319ad-c3f5-4330-884b-381691f8f8bc None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.017153 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-340319ad-c3f5-4330-884b-381691f8f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:53.021848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1383/2772] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:53.029413 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340319ad-c3f5-4330-884b-381691f8f8bc None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.035031 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.035673 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.035969 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.036713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] PATH_INFO: `/v3/projects/4adeb4ab309d44a6b3440df802f5bb25` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.060661 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4adeb4ab309d44a6b3440df802f5bb25)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.063417 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.083702 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:53.117658 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4adeb4ab309d44a6b3440df802f5bb25', 'request_id': 'req-b585a7f9-9f43-4433-84d5-c66fada35a97'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.124536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4adeb4ab309d44a6b3440df802f5bb25', 'request_id': 'req-b585a7f9-9f43-4433-84d5-c66fada35a97'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.132252 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4adeb4ab309d44a6b3440df802f5bb25', 'request_id': 'req-b585a7f9-9f43-4433-84d5-c66fada35a97'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.146651 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] The token cache is being invalidate because project 4adeb4ab309d44a6b3440df802f5bb25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:53.146939 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.147618 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-b585a7f9-9f43-4433-84d5-c66fada35a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.149144 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1390/2773] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:53 2022] DELETE /identity/v3/projects/4adeb4ab309d44a6b3440df802f5bb25 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:53.154226 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c742b-1c6c-4644-9338-b3a2f18b5dac None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.154833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c742b-1c6c-4644-9338-b3a2f18b5dac None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.155167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c742b-1c6c-4644-9338-b3a2f18b5dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.178654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ad1c742b-1c6c-4644-9338-b3a2f18b5dac None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:53.179864 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ad1c742b-1c6c-4644-9338-b3a2f18b5dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:53.230814 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:53.248530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1384/2774] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:53.256974 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfc60e-c6b8-4b33-ae7f-e63a9473fa94 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.257257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfc60e-c6b8-4b33-ae7f-e63a9473fa94 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.257529 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfc60e-c6b8-4b33-ae7f-e63a9473fa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.277388 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-74dfc60e-c6b8-4b33-ae7f-e63a9473fa94 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:53.278591 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-74dfc60e-c6b8-4b33-ae7f-e63a9473fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:53.288834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1391/2775] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:53.293061 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c742b-1c6c-4644-9338-b3a2f18b5dac None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.304100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c742b-1c6c-4644-9338-b3a2f18b5dac None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.309992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.310665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.311014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.311411 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.314958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:53.314958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.314958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.316127 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfc60e-c6b8-4b33-ae7f-e63a9473fa94 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.318326 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-74dfc60e-c6b8-4b33-ae7f-e63a9473fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:53.331350 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1385/2776] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:53.334978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfc60e-c6b8-4b33-ae7f-e63a9473fa94 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.335813 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.339976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.340655 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.340942 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.341210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] PATH_INFO: `/v3/projects/15872120c9864a7a84c7705fde43e71b` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.342577 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15872120c9864a7a84c7705fde43e71b)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.345141 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.347503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fbfab-a363-4421-8880-ae22b1073ec5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.352502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1080b-ca57-4b71-bcfb-f04743a0afdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.353146 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1080b-ca57-4b71-bcfb-f04743a0afdb admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.353984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1080b-ca57-4b71-bcfb-f04743a0afdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.354270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1080b-ca57-4b71-bcfb-f04743a0afdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.354881 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1080b-ca57-4b71-bcfb-f04743a0afdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.357515 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1080b-ca57-4b71-bcfb-f04743a0afdb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.391358 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1386/2777] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:53 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:27:53.396286 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1080b-ca57-4b71-bcfb-f04743a0afdb admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.396671 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15872120c9864a7a84c7705fde43e71b', 'request_id': 'req-600cfdcb-e32c-444b-a28d-87acd936f3f8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.403370 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15872120c9864a7a84c7705fde43e71b', 'request_id': 'req-600cfdcb-e32c-444b-a28d-87acd936f3f8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.409716 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15872120c9864a7a84c7705fde43e71b', 'request_id': 'req-600cfdcb-e32c-444b-a28d-87acd936f3f8'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.423768 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] The token cache is being invalidate because project 15872120c9864a7a84c7705fde43e71b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:53.424079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.424715 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:53.426514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1392/2778] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:53 2022] DELETE /identity/v3/projects/15872120c9864a7a84c7705fde43e71b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:27:53.463806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1080b-ca57-4b71-bcfb-f04743a0afdb admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.469032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcdc6b-e8a2-4be7-a5fc-c96c039858f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.469733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcdc6b-e8a2-4be7-a5fc-c96c039858f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.470064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcdc6b-e8a2-4be7-a5fc-c96c039858f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.470361 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcdc6b-e8a2-4be7-a5fc-c96c039858f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.471704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcdc6b-e8a2-4be7-a5fc-c96c039858f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.473740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcdc6b-e8a2-4be7-a5fc-c96c039858f0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.501908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1387/2779] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:53 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:53.503588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.505270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:53.580293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cfdcb-e32c-444b-a28d-87acd936f3f8 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.585114 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff6353-c9cd-47bc-8a63-d9f5d161822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.585734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6353-c9cd-47bc-8a63-d9f5d161822f admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.586012 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6353-c9cd-47bc-8a63-d9f5d161822f admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.586281 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6353-c9cd-47bc-8a63-d9f5d161822f admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.586930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff6353-c9cd-47bc-8a63-d9f5d161822f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.588552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff6353-c9cd-47bc-8a63-d9f5d161822f admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.600265 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1393/2780] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:53.604274 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcdc6b-e8a2-4be7-a5fc-c96c039858f0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.616828 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcdc6b-e8a2-4be7-a5fc-c96c039858f0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.622005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b7472-12b1-43b1-90d9-52210d0167e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.622619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b7472-12b1-43b1-90d9-52210d0167e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.622902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b7472-12b1-43b1-90d9-52210d0167e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.623177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b7472-12b1-43b1-90d9-52210d0167e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.623833 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b7472-12b1-43b1-90d9-52210d0167e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.625668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b7472-12b1-43b1-90d9-52210d0167e2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.636934 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1388/2781] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:53.642086 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff6353-c9cd-47bc-8a63-d9f5d161822f admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.652664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff6353-c9cd-47bc-8a63-d9f5d161822f admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.659221 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55df5d-7f05-4b01-b200-5ed72c0e67c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.660003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55df5d-7f05-4b01-b200-5ed72c0e67c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.660352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55df5d-7f05-4b01-b200-5ed72c0e67c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.660767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55df5d-7f05-4b01-b200-5ed72c0e67c6 admin admin] PATH_INFO: `/v3/projects/8b195dcecff2437caa6dada14fd2beac/users/0173a99badf04057a751b5c3a951e3f6/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.678343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55df5d-7f05-4b01-b200-5ed72c0e67c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b195dcecff2437caa6dada14fd2beac, user_id=0173a99badf04057a751b5c3a951e3f6, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.681062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55df5d-7f05-4b01-b200-5ed72c0e67c6 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.683332 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:53.696170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1394/2782] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:53 2022] PUT /identity/v3/projects/8b195dcecff2437caa6dada14fd2beac/users/0173a99badf04057a751b5c3a951e3f6/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:53.708248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d39e0a-0690-4fed-9bf7-b6c8dfb637c3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.708649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d39e0a-0690-4fed-9bf7-b6c8dfb637c3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.708957 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d39e0a-0690-4fed-9bf7-b6c8dfb637c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.738428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-49d39e0a-0690-4fed-9bf7-b6c8dfb637c3 None None] MFA Rules not processed for user `0173a99badf04057a751b5c3a951e3f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:53.739661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-49d39e0a-0690-4fed-9bf7-b6c8dfb637c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:53.808056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1389/2783] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:53.812676 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55df5d-7f05-4b01-b200-5ed72c0e67c6 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.885707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55df5d-7f05-4b01-b200-5ed72c0e67c6 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.889759 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d24f9-b0bf-40a7-a2cb-595a3a6d02f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:53.891075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d24f9-b0bf-40a7-a2cb-595a3a6d02f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:53.891410 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d24f9-b0bf-40a7-a2cb-595a3a6d02f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:53.891645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d24f9-b0bf-40a7-a2cb-595a3a6d02f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:53.892237 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d24f9-b0bf-40a7-a2cb-595a3a6d02f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:53.894203 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d24f9-b0bf-40a7-a2cb-595a3a6d02f0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:53.929225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1395/2784] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:53 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:53.933609 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d39e0a-0690-4fed-9bf7-b6c8dfb637c3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:53.945619 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d39e0a-0690-4fed-9bf7-b6c8dfb637c3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:53.965537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:54.005607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23281c7c-902b-470e-86ad-19b5e10567f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.006206 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23281c7c-902b-470e-86ad-19b5e10567f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.006484 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23281c7c-902b-470e-86ad-19b5e10567f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.006755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23281c7c-902b-470e-86ad-19b5e10567f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.007393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23281c7c-902b-470e-86ad-19b5e10567f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.009341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23281c7c-902b-470e-86ad-19b5e10567f2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.033169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1390/2785] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:53 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:54.037313 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d24f9-b0bf-40a7-a2cb-595a3a6d02f0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.048515 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d24f9-b0bf-40a7-a2cb-595a3a6d02f0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.053646 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2ea52-484e-4794-b7be-157b93b32bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.054272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2ea52-484e-4794-b7be-157b93b32bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.054557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2ea52-484e-4794-b7be-157b93b32bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.054834 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2ea52-484e-4794-b7be-157b93b32bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.055496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2ea52-484e-4794-b7be-157b93b32bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.057446 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2ea52-484e-4794-b7be-157b93b32bd9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.070945 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1396/2786] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:54.076761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23281c7c-902b-470e-86ad-19b5e10567f2 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.087021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23281c7c-902b-470e-86ad-19b5e10567f2 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.092686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acf6d3-6fda-4447-a179-61c254e1bcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.093434 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acf6d3-6fda-4447-a179-61c254e1bcb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.093900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acf6d3-6fda-4447-a179-61c254e1bcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.094173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acf6d3-6fda-4447-a179-61c254e1bcb4 admin admin] PATH_INFO: `/v3/projects/253c4ef6ba5642e08724fac2890c8d3f/users/720d20c0854e47818257fa9b02caa830/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.111681 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acf6d3-6fda-4447-a179-61c254e1bcb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253c4ef6ba5642e08724fac2890c8d3f, user_id=720d20c0854e47818257fa9b02caa830, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.114371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acf6d3-6fda-4447-a179-61c254e1bcb4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.132305 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1391/2787] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:54 2022] PUT /identity/v3/projects/253c4ef6ba5642e08724fac2890c8d3f/users/720d20c0854e47818257fa9b02caa830/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:54.136650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2ea52-484e-4794-b7be-157b93b32bd9 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.154175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2ea52-484e-4794-b7be-157b93b32bd9 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.214623 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea0931-bbac-430e-a146-bf29499d08b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.215238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea0931-bbac-430e-a146-bf29499d08b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.215543 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea0931-bbac-430e-a146-bf29499d08b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.215821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea0931-bbac-430e-a146-bf29499d08b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.216472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea0931-bbac-430e-a146-bf29499d08b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.218175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea0931-bbac-430e-a146-bf29499d08b9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.228793 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1397/2788] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:54.233213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acf6d3-6fda-4447-a179-61c254e1bcb4 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.244618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acf6d3-6fda-4447-a179-61c254e1bcb4 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.249519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.250759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.251194 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.251464 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] PATH_INFO: `/v3/projects/253c4ef6ba5642e08724fac2890c8d3f/users/720d20c0854e47818257fa9b02caa830/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.253537 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253c4ef6ba5642e08724fac2890c8d3f, user_id=720d20c0854e47818257fa9b02caa830, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.256186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.268873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1392/2789] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:54 2022] PUT /identity/v3/projects/253c4ef6ba5642e08724fac2890c8d3f/users/720d20c0854e47818257fa9b02caa830/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:54.279351 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7f744-a794-497c-9cd7-c675a0299c55 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.279910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7f744-a794-497c-9cd7-c675a0299c55 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.280188 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7f744-a794-497c-9cd7-c675a0299c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.311309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-aee7f744-a794-497c-9cd7-c675a0299c55 None None] MFA Rules not processed for user `720d20c0854e47818257fa9b02caa830`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:54.312540 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-aee7f744-a794-497c-9cd7-c675a0299c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:54.366736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:54.388224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1398/2790] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:54.401701 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.403563 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:54.468513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49049c1-4998-4a59-9bd2-3fbb975ecdc7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.472421 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4899eb-8008-49e8-9f55-be156eb51341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.473050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4899eb-8008-49e8-9f55-be156eb51341 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.473354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4899eb-8008-49e8-9f55-be156eb51341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.473624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4899eb-8008-49e8-9f55-be156eb51341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.483117 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4899eb-8008-49e8-9f55-be156eb51341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.484635 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4899eb-8008-49e8-9f55-be156eb51341 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:54.494546 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1393/2791] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:27:54.891184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7f744-a794-497c-9cd7-c675a0299c55 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:54.954286 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7f744-a794-497c-9cd7-c675a0299c55 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:54.958727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:54.959339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:54.959621 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:54.959885 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] PATH_INFO: `/v3/users/0173a99badf04057a751b5c3a951e3f6` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:54.961057 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0173a99badf04057a751b5c3a951e3f6)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:54.962956 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.007896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0173a99badf04057a751b5c3a951e3f6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.014870 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0173a99badf04057a751b5c3a951e3f6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.015648 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0173a99badf04057a751b5c3a951e3f6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.027437 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0173a99badf04057a751b5c3a951e3f6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.028081 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0173a99badf04057a751b5c3a951e3f6'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.041670 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1399/2792] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:54 2022] DELETE /identity/v3/users/0173a99badf04057a751b5c3a951e3f6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:55.046198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee444cd-a863-4efe-9a96-9aaefa1af190 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.047398 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee444cd-a863-4efe-9a96-9aaefa1af190 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.047500 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee444cd-a863-4efe-9a96-9aaefa1af190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.066749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8ee444cd-a863-4efe-9a96-9aaefa1af190 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:55.067962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8ee444cd-a863-4efe-9a96-9aaefa1af190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:55.139588 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1394/2793] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:55.149594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.151039 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:55.210809 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92d04f-ce2c-48e5-815c-822bca3fcf2a admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.214452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200d92f-4941-4e2b-ab2b-d7968f0c39b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.215058 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d92f-4941-4e2b-ab2b-d7968f0c39b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.215336 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d92f-4941-4e2b-ab2b-d7968f0c39b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.215606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d92f-4941-4e2b-ab2b-d7968f0c39b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.226061 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200d92f-4941-4e2b-ab2b-d7968f0c39b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.227558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200d92f-4941-4e2b-ab2b-d7968f0c39b7 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.238047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1400/2794] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:55.513477 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee444cd-a863-4efe-9a96-9aaefa1af190 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.514917 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8ee444cd-a863-4efe-9a96-9aaefa1af190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:55.527992 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee444cd-a863-4efe-9a96-9aaefa1af190 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.533726 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.534401 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.534684 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.534958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] PATH_INFO: `/v3/projects/8b195dcecff2437caa6dada14fd2beac` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.536135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b195dcecff2437caa6dada14fd2beac)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.538203 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.549968 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:55.586941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b195dcecff2437caa6dada14fd2beac', 'request_id': 'req-6469829d-03d9-45e4-925e-104dd28f294f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.595363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b195dcecff2437caa6dada14fd2beac', 'request_id': 'req-6469829d-03d9-45e4-925e-104dd28f294f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.601607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b195dcecff2437caa6dada14fd2beac', 'request_id': 'req-6469829d-03d9-45e4-925e-104dd28f294f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.611704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] The token cache is being invalidate because project 8b195dcecff2437caa6dada14fd2beac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:55.611987 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.612777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.614266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1395/2795] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:55 2022] DELETE /identity/v3/projects/8b195dcecff2437caa6dada14fd2beac => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:55.618355 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200d92f-4941-4e2b-ab2b-d7968f0c39b7 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:55.627789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200d92f-4941-4e2b-ab2b-d7968f0c39b7 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:55.635341 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:55.686385 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:55.686930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:55.687499 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:55.688210 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] PATH_INFO: `/v3/users/720d20c0854e47818257fa9b02caa830` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:55.689320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=720d20c0854e47818257fa9b02caa830)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:55.691244 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:55.736982 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720d20c0854e47818257fa9b02caa830'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.743773 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720d20c0854e47818257fa9b02caa830'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.744427 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720d20c0854e47818257fa9b02caa830'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.756461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720d20c0854e47818257fa9b02caa830'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.757115 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3114dc6c-09ce-4e2e-942e-50e004fe6193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720d20c0854e47818257fa9b02caa830'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:55.772820 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1401/2796] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:27:55 2022] DELETE /identity/v3/users/720d20c0854e47818257fa9b02caa830 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:56.053298 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.054782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:56.070341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469829d-03d9-45e4-925e-104dd28f294f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.130941 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.131608 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.131984 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.132287 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] PATH_INFO: `/v3/projects/253c4ef6ba5642e08724fac2890c8d3f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.134577 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253c4ef6ba5642e08724fac2890c8d3f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.135541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.194176 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253c4ef6ba5642e08724fac2890c8d3f', 'request_id': 'req-fac33031-3db3-40ff-bf76-7671095b4054'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:56.204090 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253c4ef6ba5642e08724fac2890c8d3f', 'request_id': 'req-fac33031-3db3-40ff-bf76-7671095b4054'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:56.211173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253c4ef6ba5642e08724fac2890c8d3f', 'request_id': 'req-fac33031-3db3-40ff-bf76-7671095b4054'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:56.222095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] The token cache is being invalidate because project 253c4ef6ba5642e08724fac2890c8d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:27:56.222453 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:56.223057 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fac33031-3db3-40ff-bf76-7671095b4054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:27:56.224652 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1396/2797] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:27:56 2022] DELETE /identity/v3/projects/253c4ef6ba5642e08724fac2890c8d3f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:56.229933 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb099f-0d52-4777-b095-cf936c432ac7 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.230478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb099f-0d52-4777-b095-cf936c432ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.230478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb099f-0d52-4777-b095-cf936c432ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.252824 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-addb099f-0d52-4777-b095-cf936c432ac7 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:56.254064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-addb099f-0d52-4777-b095-cf936c432ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:56.319806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1402/2798] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:56.328793 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc71660-b862-4472-9312-70dfdc7a2310 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.329073 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc71660-b862-4472-9312-70dfdc7a2310 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.329341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc71660-b862-4472-9312-70dfdc7a2310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.348961 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8fc71660-b862-4472-9312-70dfdc7a2310 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:56.350148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8fc71660-b862-4472-9312-70dfdc7a2310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:56.357848 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1397/2799] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:56.362634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb099f-0d52-4777-b095-cf936c432ac7 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.372382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb099f-0d52-4777-b095-cf936c432ac7 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.378367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.378962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.379245 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.379514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.380161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:27:56.380464 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.382100 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.393823 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1403/2800] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:27:56.397998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc71660-b862-4472-9312-70dfdc7a2310 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.409366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc71660-b862-4472-9312-70dfdc7a2310 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.414025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5127039-2953-47ef-b428-8e63504d35e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.414618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5127039-2953-47ef-b428-8e63504d35e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.414897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5127039-2953-47ef-b428-8e63504d35e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.415167 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5127039-2953-47ef-b428-8e63504d35e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.415801 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5127039-2953-47ef-b428-8e63504d35e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.417767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5127039-2953-47ef-b428-8e63504d35e9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.446394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1398/2801] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:56 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:56.450857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.461234 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c44f2-fb2c-4242-b98b-7a33e5213b33 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.475414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:56.525359 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7ae72-8079-4c90-883d-506dd15c5b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.526174 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ae72-8079-4c90-883d-506dd15c5b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.526456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ae72-8079-4c90-883d-506dd15c5b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.526726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ae72-8079-4c90-883d-506dd15c5b3a admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.527371 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7ae72-8079-4c90-883d-506dd15c5b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.529668 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7ae72-8079-4c90-883d-506dd15c5b3a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.554925 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1404/2802] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:56 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:56.559280 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5127039-2953-47ef-b428-8e63504d35e9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.564687 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:56.571526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5127039-2953-47ef-b428-8e63504d35e9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.577181 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31d571-7e47-4060-8618-8583c7e4b51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.577798 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31d571-7e47-4060-8618-8583c7e4b51e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.578086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31d571-7e47-4060-8618-8583c7e4b51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.578355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31d571-7e47-4060-8618-8583c7e4b51e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.579196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31d571-7e47-4060-8618-8583c7e4b51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.580841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31d571-7e47-4060-8618-8583c7e4b51e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.593104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1399/2803] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:27:56.597756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7ae72-8079-4c90-883d-506dd15c5b3a admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.608088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7ae72-8079-4c90-883d-506dd15c5b3a admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.616260 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791844d-73de-424e-845b-622060e83f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.616916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791844d-73de-424e-845b-622060e83f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.617202 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791844d-73de-424e-845b-622060e83f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.617471 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791844d-73de-424e-845b-622060e83f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.618118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791844d-73de-424e-845b-622060e83f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.619739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791844d-73de-424e-845b-622060e83f18 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.631664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1405/2804] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:56.636349 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31d571-7e47-4060-8618-8583c7e4b51e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.646830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31d571-7e47-4060-8618-8583c7e4b51e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.651791 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70afd2a-a55f-4af9-a564-ad7ad0e9f978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.652440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70afd2a-a55f-4af9-a564-ad7ad0e9f978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.652750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70afd2a-a55f-4af9-a564-ad7ad0e9f978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.653025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70afd2a-a55f-4af9-a564-ad7ad0e9f978 admin admin] PATH_INFO: `/v3/projects/f887d0ca9d3b4618958a466343b20b13/users/339d24731ae546339943ce7ae42c38af/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.669842 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70afd2a-a55f-4af9-a564-ad7ad0e9f978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f887d0ca9d3b4618958a466343b20b13, user_id=339d24731ae546339943ce7ae42c38af, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.673380 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70afd2a-a55f-4af9-a564-ad7ad0e9f978 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.686602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1400/2805] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:56 2022] PUT /identity/v3/projects/f887d0ca9d3b4618958a466343b20b13/users/339d24731ae546339943ce7ae42c38af/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:56.697755 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403b36a-cbdb-49a1-980f-10afadd5899d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.698047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403b36a-cbdb-49a1-980f-10afadd5899d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.698310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403b36a-cbdb-49a1-980f-10afadd5899d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.729346 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f403b36a-cbdb-49a1-980f-10afadd5899d None None] MFA Rules not processed for user `339d24731ae546339943ce7ae42c38af`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:56.730510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f403b36a-cbdb-49a1-980f-10afadd5899d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:56.798653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1406/2806] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:56.802971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70afd2a-a55f-4af9-a564-ad7ad0e9f978 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.865152 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70afd2a-a55f-4af9-a564-ad7ad0e9f978 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.869905 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a0fcb-fef9-44d8-bc41-931982930146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.870502 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0fcb-fef9-44d8-bc41-931982930146 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.870785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0fcb-fef9-44d8-bc41-931982930146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.871062 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0fcb-fef9-44d8-bc41-931982930146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.871700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a0fcb-fef9-44d8-bc41-931982930146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.873646 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a0fcb-fef9-44d8-bc41-931982930146 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.899979 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1401/2807] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:27:56 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:56.904215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403b36a-cbdb-49a1-980f-10afadd5899d None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:56.915876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403b36a-cbdb-49a1-980f-10afadd5899d None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:56.972356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96f330-f78c-4f01-a12f-2248454138fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:56.973024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96f330-f78c-4f01-a12f-2248454138fd admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:56.973302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96f330-f78c-4f01-a12f-2248454138fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:56.973574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96f330-f78c-4f01-a12f-2248454138fd admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:56.974215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96f330-f78c-4f01-a12f-2248454138fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:56.976157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96f330-f78c-4f01-a12f-2248454138fd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:56.997276 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1407/2808] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:27:56 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:27:57.001344 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a0fcb-fef9-44d8-bc41-931982930146 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:57.011770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a0fcb-fef9-44d8-bc41-931982930146 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:57.017135 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad57e11-c0b5-4e70-905d-7e9dd407b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:57.017732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad57e11-c0b5-4e70-905d-7e9dd407b87b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.018014 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad57e11-c0b5-4e70-905d-7e9dd407b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.018283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad57e11-c0b5-4e70-905d-7e9dd407b87b admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.018918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad57e11-c0b5-4e70-905d-7e9dd407b87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:57.020511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad57e11-c0b5-4e70-905d-7e9dd407b87b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:57.032085 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1402/2809] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:57.036187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96f330-f78c-4f01-a12f-2248454138fd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:57.046207 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96f330-f78c-4f01-a12f-2248454138fd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:57.050944 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292b3a-2e1e-4a38-93e5-5c15bc756ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:57.051587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292b3a-2e1e-4a38-93e5-5c15bc756ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.051867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292b3a-2e1e-4a38-93e5-5c15bc756ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.052170 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292b3a-2e1e-4a38-93e5-5c15bc756ccf admin admin] PATH_INFO: `/v3/projects/97e73f226db242f48bb7a36106f2713a/users/82983ada0b5c4aaeae83d390c5a0c41b/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.065500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:57.068683 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292b3a-2e1e-4a38-93e5-5c15bc756ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e73f226db242f48bb7a36106f2713a, user_id=82983ada0b5c4aaeae83d390c5a0c41b, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:57.071799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292b3a-2e1e-4a38-93e5-5c15bc756ccf admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:57.085734 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1408/2810] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:57 2022] PUT /identity/v3/projects/97e73f226db242f48bb7a36106f2713a/users/82983ada0b5c4aaeae83d390c5a0c41b/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:57.089675 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad57e11-c0b5-4e70-905d-7e9dd407b87b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:57.100393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad57e11-c0b5-4e70-905d-7e9dd407b87b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:57.158963 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb0388-337e-4776-abe2-084a7f734671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:57.159571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb0388-337e-4776-abe2-084a7f734671 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.159853 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb0388-337e-4776-abe2-084a7f734671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.160125 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb0388-337e-4776-abe2-084a7f734671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.160811 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb0388-337e-4776-abe2-084a7f734671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:57.162407 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb0388-337e-4776-abe2-084a7f734671 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:57.172764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1403/2811] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:27:57.177101 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292b3a-2e1e-4a38-93e5-5c15bc756ccf admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:57.187935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292b3a-2e1e-4a38-93e5-5c15bc756ccf admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:57.192795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:57.193747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.194310 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.194848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] PATH_INFO: `/v3/projects/97e73f226db242f48bb7a36106f2713a/users/82983ada0b5c4aaeae83d390c5a0c41b/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.198335 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e73f226db242f48bb7a36106f2713a, user_id=82983ada0b5c4aaeae83d390c5a0c41b, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:57.203612 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:57.215211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1409/2812] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:27:57 2022] PUT /identity/v3/projects/97e73f226db242f48bb7a36106f2713a/users/82983ada0b5c4aaeae83d390c5a0c41b/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:27:57.226920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca502df9-f798-4dec-8f23-34e4f0f5183d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.227208 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca502df9-f798-4dec-8f23-34e4f0f5183d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.227482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca502df9-f798-4dec-8f23-34e4f0f5183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.254505 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ca502df9-f798-4dec-8f23-34e4f0f5183d None None] MFA Rules not processed for user `82983ada0b5c4aaeae83d390c5a0c41b`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:57.255630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ca502df9-f798-4dec-8f23-34e4f0f5183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:57.282458 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:57.323607 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1404/2813] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:27:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:57.336587 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:57.338392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:57.397367 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc6122-d820-4a36-a7b5-46237a3dab4a admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:57.401004 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ce962-a255-4711-8ef2-e1d673a1a621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:57.401599 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce962-a255-4711-8ef2-e1d673a1a621 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.401876 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce962-a255-4711-8ef2-e1d673a1a621 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.402148 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ce962-a255-4711-8ef2-e1d673a1a621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.412637 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ce962-a255-4711-8ef2-e1d673a1a621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:57.414076 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ce962-a255-4711-8ef2-e1d673a1a621 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:57.423948 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1410/2814] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:57.500506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca502df9-f798-4dec-8f23-34e4f0f5183d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:57.510214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca502df9-f798-4dec-8f23-34e4f0f5183d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:57.513680 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfef842-cd78-457e-b904-15a53e460416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:57.514265 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfef842-cd78-457e-b904-15a53e460416 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:57.514541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfef842-cd78-457e-b904-15a53e460416 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:57.514809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfef842-cd78-457e-b904-15a53e460416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:57.570709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfef842-cd78-457e-b904-15a53e460416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:57.572182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfef842-cd78-457e-b904-15a53e460416 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:57.581232 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1405/2815] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:00.340329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ce962-a255-4711-8ef2-e1d673a1a621 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:00.403015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ce962-a255-4711-8ef2-e1d673a1a621 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:00.408459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:00.409105 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.409387 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.409667 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] PATH_INFO: `/v3/users/339d24731ae546339943ce7ae42c38af` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.410789 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=339d24731ae546339943ce7ae42c38af)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:00.412934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:00.459401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '339d24731ae546339943ce7ae42c38af'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.466795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '339d24731ae546339943ce7ae42c38af'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.467503 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '339d24731ae546339943ce7ae42c38af'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.479943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '339d24731ae546339943ce7ae42c38af'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.480619 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '339d24731ae546339943ce7ae42c38af'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:00.490117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:00.494935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1411/2816] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:00 2022] DELETE /identity/v3/users/339d24731ae546339943ce7ae42c38af => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:00.499630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9248-d307-469e-ad6f-f8053eccb258 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.500340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9248-d307-469e-ad6f-f8053eccb258 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.500340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9248-d307-469e-ad6f-f8053eccb258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.520273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-709c9248-d307-469e-ad6f-f8053eccb258 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:00.521516 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-709c9248-d307-469e-ad6f-f8053eccb258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:00.581198 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1406/2817] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:28:00.590964 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:00.592400 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:00.650320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c308d-3b4f-4f4d-b314-fe336b596bca admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:00.654106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739138d9-af82-43b1-9a77-45fc75a6ca51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:00.654910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739138d9-af82-43b1-9a77-45fc75a6ca51 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.655190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739138d9-af82-43b1-9a77-45fc75a6ca51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.655457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739138d9-af82-43b1-9a77-45fc75a6ca51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.665698 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739138d9-af82-43b1-9a77-45fc75a6ca51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:00.667178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739138d9-af82-43b1-9a77-45fc75a6ca51 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:00.676923 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1412/2818] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:00.947810 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c9248-d307-469e-ad6f-f8053eccb258 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:00.949375 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-709c9248-d307-469e-ad6f-f8053eccb258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:00.962139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c9248-d307-469e-ad6f-f8053eccb258 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:00.967540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:00.971573 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:00.972304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:00.972304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] PATH_INFO: `/v3/projects/f887d0ca9d3b4618958a466343b20b13` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:00.973337 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f887d0ca9d3b4618958a466343b20b13)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:00.975325 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.022102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f887d0ca9d3b4618958a466343b20b13', 'request_id': 'req-f2cde231-63ea-420b-a958-0b8aee5880d1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.029611 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f887d0ca9d3b4618958a466343b20b13', 'request_id': 'req-f2cde231-63ea-420b-a958-0b8aee5880d1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.031503 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:01.038345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f887d0ca9d3b4618958a466343b20b13', 'request_id': 'req-f2cde231-63ea-420b-a958-0b8aee5880d1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.049799 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] The token cache is being invalidate because project f887d0ca9d3b4618958a466343b20b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:01.050089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.051207 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.052737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1407/2819] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:00 2022] DELETE /identity/v3/projects/f887d0ca9d3b4618958a466343b20b13 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:01.056841 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739138d9-af82-43b1-9a77-45fc75a6ca51 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:01.067338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739138d9-af82-43b1-9a77-45fc75a6ca51 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:01.123993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:01.124691 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.124967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.125242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] PATH_INFO: `/v3/users/82983ada0b5c4aaeae83d390c5a0c41b` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.126709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] RBAC: Authorizing `identity:delete_user(user_id=82983ada0b5c4aaeae83d390c5a0c41b)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:01.128353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.176851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82983ada0b5c4aaeae83d390c5a0c41b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.183606 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82983ada0b5c4aaeae83d390c5a0c41b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.184264 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82983ada0b5c4aaeae83d390c5a0c41b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.196613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82983ada0b5c4aaeae83d390c5a0c41b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.197194 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-e07c9b22-35cf-455c-a5a4-eea8c565a72d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82983ada0b5c4aaeae83d390c5a0c41b'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.209792 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1413/2820] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:01 2022] DELETE /identity/v3/users/82983ada0b5c4aaeae83d390c5a0c41b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:01.489859 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:01.491455 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:01.507308 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde231-63ea-420b-a958-0b8aee5880d1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:01.564334 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:01.564988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.565264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.565526 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] PATH_INFO: `/v3/projects/97e73f226db242f48bb7a36106f2713a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.566627 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e73f226db242f48bb7a36106f2713a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:01.568519 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.613316 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e73f226db242f48bb7a36106f2713a', 'request_id': 'req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.620426 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e73f226db242f48bb7a36106f2713a', 'request_id': 'req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.627170 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e73f226db242f48bb7a36106f2713a', 'request_id': 'req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.637359 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] The token cache is being invalidate because project 97e73f226db242f48bb7a36106f2713a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:01.637645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.638270 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d27e8331-cbf1-45a3-9514-2dc914a3b9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:01.639734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1408/2821] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:01 2022] DELETE /identity/v3/projects/97e73f226db242f48bb7a36106f2713a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:01.645001 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28152aac-3c7e-4a23-9fc9-eac5de6c44d6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.645481 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28152aac-3c7e-4a23-9fc9-eac5de6c44d6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.645583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28152aac-3c7e-4a23-9fc9-eac5de6c44d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.665946 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-28152aac-3c7e-4a23-9fc9-eac5de6c44d6 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:01.667157 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-28152aac-3c7e-4a23-9fc9-eac5de6c44d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:01.685073 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:01.729171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1414/2822] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:01.738476 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8dc9a-dafd-4721-9751-f86342141b68 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.738913 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8dc9a-dafd-4721-9751-f86342141b68 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.739157 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8dc9a-dafd-4721-9751-f86342141b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.759441 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-45f8dc9a-dafd-4721-9751-f86342141b68 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:01.760925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-45f8dc9a-dafd-4721-9751-f86342141b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:01.769594 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1409/2823] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:01.773931 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28152aac-3c7e-4a23-9fc9-eac5de6c44d6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:01.784803 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28152aac-3c7e-4a23-9fc9-eac5de6c44d6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:01.789293 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:01.789884 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.790166 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.790430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.791075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:01.791375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:01.793008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.805440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1415/2824] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:01.809703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8dc9a-dafd-4721-9751-f86342141b68 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:01.819370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8dc9a-dafd-4721-9751-f86342141b68 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:01.824024 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e518aae-2f19-455d-8e58-3f7669430985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:01.824642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518aae-2f19-455d-8e58-3f7669430985 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.824925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518aae-2f19-455d-8e58-3f7669430985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.825199 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518aae-2f19-455d-8e58-3f7669430985 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.825825 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e518aae-2f19-455d-8e58-3f7669430985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:01.827740 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e518aae-2f19-455d-8e58-3f7669430985 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.856103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1410/2825] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:01 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:01.860770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:01.871040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37325c-bfeb-4a83-9ce4-91c390828994 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:01.927294 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74529fb-64d7-4d1b-a9b9-e594727aa342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:01.927904 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74529fb-64d7-4d1b-a9b9-e594727aa342 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.928186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74529fb-64d7-4d1b-a9b9-e594727aa342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.928457 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74529fb-64d7-4d1b-a9b9-e594727aa342 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.929126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74529fb-64d7-4d1b-a9b9-e594727aa342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:01.931079 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74529fb-64d7-4d1b-a9b9-e594727aa342 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.958180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1416/2826] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:01 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:01.962539 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e518aae-2f19-455d-8e58-3f7669430985 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:01.973958 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e518aae-2f19-455d-8e58-3f7669430985 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:01.978921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd13ed2-c420-4758-bb97-dcedc0604986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:01.979527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd13ed2-c420-4758-bb97-dcedc0604986 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:01.979809 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd13ed2-c420-4758-bb97-dcedc0604986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:01.980083 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd13ed2-c420-4758-bb97-dcedc0604986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:01.980773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd13ed2-c420-4758-bb97-dcedc0604986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:01.982377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd13ed2-c420-4758-bb97-dcedc0604986 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:01.993475 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1411/2827] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:01.997523 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74529fb-64d7-4d1b-a9b9-e594727aa342 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.008024 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74529fb-64d7-4d1b-a9b9-e594727aa342 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.012719 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66df88-2cbd-41eb-abb3-5a2b47a5e34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.013319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66df88-2cbd-41eb-abb3-5a2b47a5e34e admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.013602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66df88-2cbd-41eb-abb3-5a2b47a5e34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.013877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66df88-2cbd-41eb-abb3-5a2b47a5e34e admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.014524 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66df88-2cbd-41eb-abb3-5a2b47a5e34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.016411 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66df88-2cbd-41eb-abb3-5a2b47a5e34e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.026981 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1417/2828] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:02.032160 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd13ed2-c420-4758-bb97-dcedc0604986 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.042304 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd13ed2-c420-4758-bb97-dcedc0604986 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.047621 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73b11a-f082-4106-8a20-3e6f2d5af185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.048269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73b11a-f082-4106-8a20-3e6f2d5af185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.048556 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73b11a-f082-4106-8a20-3e6f2d5af185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.048860 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73b11a-f082-4106-8a20-3e6f2d5af185 admin admin] PATH_INFO: `/v3/projects/f9b53bb2395c4f8abb2a038464a17d3b/users/9ab50e338b9540b68665a3bb83e063ec/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.063678 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73b11a-f082-4106-8a20-3e6f2d5af185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b53bb2395c4f8abb2a038464a17d3b, user_id=9ab50e338b9540b68665a3bb83e063ec, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.066431 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73b11a-f082-4106-8a20-3e6f2d5af185 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.079297 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1412/2829] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:02 2022] PUT /identity/v3/projects/f9b53bb2395c4f8abb2a038464a17d3b/users/9ab50e338b9540b68665a3bb83e063ec/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:02.089700 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860ac12-9e5a-4989-9c8f-e0438444fe3e None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.089987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860ac12-9e5a-4989-9c8f-e0438444fe3e None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.090275 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860ac12-9e5a-4989-9c8f-e0438444fe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.119957 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f860ac12-9e5a-4989-9c8f-e0438444fe3e None None] MFA Rules not processed for user `9ab50e338b9540b68665a3bb83e063ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:02.121235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f860ac12-9e5a-4989-9c8f-e0438444fe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:02.187805 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1418/2830] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:02.192449 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73b11a-f082-4106-8a20-3e6f2d5af185 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.206064 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:02.261937 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73b11a-f082-4106-8a20-3e6f2d5af185 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.267919 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab021-7bfd-4a2a-85ad-17a6b8eb9cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.268522 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab021-7bfd-4a2a-85ad-17a6b8eb9cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.268838 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab021-7bfd-4a2a-85ad-17a6b8eb9cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.269107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab021-7bfd-4a2a-85ad-17a6b8eb9cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.269751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ab021-7bfd-4a2a-85ad-17a6b8eb9cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.271661 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ab021-7bfd-4a2a-85ad-17a6b8eb9cf5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.299259 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1413/2831] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:28:02.303305 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860ac12-9e5a-4989-9c8f-e0438444fe3e None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.314487 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860ac12-9e5a-4989-9c8f-e0438444fe3e None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.326502 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:02.371950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38582e-1a43-4ec5-a60a-8268ad510a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.372714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38582e-1a43-4ec5-a60a-8268ad510a6a admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.372999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38582e-1a43-4ec5-a60a-8268ad510a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.373272 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38582e-1a43-4ec5-a60a-8268ad510a6a admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.373910 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c38582e-1a43-4ec5-a60a-8268ad510a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.376236 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c38582e-1a43-4ec5-a60a-8268ad510a6a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.400181 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1419/2832] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:02.404670 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab021-7bfd-4a2a-85ad-17a6b8eb9cf5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.415806 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab021-7bfd-4a2a-85ad-17a6b8eb9cf5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.420301 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52f3b9-e7c8-4929-8158-4867d47272f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.420915 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f3b9-e7c8-4929-8158-4867d47272f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.421192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f3b9-e7c8-4929-8158-4867d47272f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.421508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f3b9-e7c8-4929-8158-4867d47272f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.422139 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52f3b9-e7c8-4929-8158-4867d47272f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.423728 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52f3b9-e7c8-4929-8158-4867d47272f1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.435585 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1414/2833] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:02.440010 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38582e-1a43-4ec5-a60a-8268ad510a6a admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.450218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38582e-1a43-4ec5-a60a-8268ad510a6a admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.455478 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f129d7-2891-45f4-9a4f-e68c1b9455c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.456118 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f129d7-2891-45f4-9a4f-e68c1b9455c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.456394 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f129d7-2891-45f4-9a4f-e68c1b9455c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.456701 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f129d7-2891-45f4-9a4f-e68c1b9455c9 admin admin] PATH_INFO: `/v3/projects/240bc122474e4a198bdd32a14d1d0e5e/users/5f941368f55947b0a2c83a76a9393ad0/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.472256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f129d7-2891-45f4-9a4f-e68c1b9455c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240bc122474e4a198bdd32a14d1d0e5e, user_id=5f941368f55947b0a2c83a76a9393ad0, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.474968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f129d7-2891-45f4-9a4f-e68c1b9455c9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.487472 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1420/2834] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:02 2022] PUT /identity/v3/projects/240bc122474e4a198bdd32a14d1d0e5e/users/5f941368f55947b0a2c83a76a9393ad0/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:02.491654 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52f3b9-e7c8-4929-8158-4867d47272f1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.503580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52f3b9-e7c8-4929-8158-4867d47272f1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.561154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48574194-f9fb-43be-93ea-30ba2f1d2a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.562462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48574194-f9fb-43be-93ea-30ba2f1d2a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.562462 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48574194-f9fb-43be-93ea-30ba2f1d2a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.562686 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48574194-f9fb-43be-93ea-30ba2f1d2a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.563330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48574194-f9fb-43be-93ea-30ba2f1d2a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.564940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48574194-f9fb-43be-93ea-30ba2f1d2a33 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.574575 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1415/2835] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:28:02.579476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f129d7-2891-45f4-9a4f-e68c1b9455c9 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.590022 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f129d7-2891-45f4-9a4f-e68c1b9455c9 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.594706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.595349 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.595633 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.595908 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] PATH_INFO: `/v3/projects/240bc122474e4a198bdd32a14d1d0e5e/users/5f941368f55947b0a2c83a76a9393ad0/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.597873 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240bc122474e4a198bdd32a14d1d0e5e, user_id=5f941368f55947b0a2c83a76a9393ad0, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.600588 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.611582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1421/2836] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:02 2022] PUT /identity/v3/projects/240bc122474e4a198bdd32a14d1d0e5e/users/5f941368f55947b0a2c83a76a9393ad0/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:02.628807 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e951702-c895-41e2-8d39-ce7de2d16274 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.629087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e951702-c895-41e2-8d39-ce7de2d16274 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.629355 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e951702-c895-41e2-8d39-ce7de2d16274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.658269 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6e951702-c895-41e2-8d39-ce7de2d16274 None None] MFA Rules not processed for user `5f941368f55947b0a2c83a76a9393ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:02.659489 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6e951702-c895-41e2-8d39-ce7de2d16274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:02.729384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1416/2837] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:02.740204 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:02.741990 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:02.801500 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c9e45-ad97-4c0f-a579-7ea53f521fb9 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:02.805299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e403e3e-476e-4cf5-aefc-4b64d9590f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:02.805905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e403e3e-476e-4cf5-aefc-4b64d9590f2c service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:02.806215 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e403e3e-476e-4cf5-aefc-4b64d9590f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:02.806490 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e403e3e-476e-4cf5-aefc-4b64d9590f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:02.862617 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e403e3e-476e-4cf5-aefc-4b64d9590f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:02.863986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e403e3e-476e-4cf5-aefc-4b64d9590f2c service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:02.873196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1422/2838] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:07.492587 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e951702-c895-41e2-8d39-ce7de2d16274 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:07.502246 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e951702-c895-41e2-8d39-ce7de2d16274 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:07.505767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefcba2-4ad2-4596-b6a8-6f6d2720f96f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:07.506357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefcba2-4ad2-4596-b6a8-6f6d2720f96f service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:07.506637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefcba2-4ad2-4596-b6a8-6f6d2720f96f service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:07.506907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefcba2-4ad2-4596-b6a8-6f6d2720f96f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:07.512850 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:07.518340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefcba2-4ad2-4596-b6a8-6f6d2720f96f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:07.520600 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefcba2-4ad2-4596-b6a8-6f6d2720f96f service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:07.531746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1417/2839] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:12.869015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:17.526128 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:29.429914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e403e3e-476e-4cf5-aefc-4b64d9590f2c service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:29.493997 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e403e3e-476e-4cf5-aefc-4b64d9590f2c service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:29.498452 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:29.499074 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:29.499353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:29.499624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] PATH_INFO: `/v3/users/9ab50e338b9540b68665a3bb83e063ec` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:29.501340 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab50e338b9540b68665a3bb83e063ec)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:29.503338 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:29.552123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab50e338b9540b68665a3bb83e063ec'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:29.560763 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab50e338b9540b68665a3bb83e063ec'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:29.561535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab50e338b9540b68665a3bb83e063ec'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:29.574102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab50e338b9540b68665a3bb83e063ec'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:29.574760 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab50e338b9540b68665a3bb83e063ec'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:29.589109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1423/2840] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:29 2022] DELETE /identity/v3/users/9ab50e338b9540b68665a3bb83e063ec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:29.593796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efbab4-91e5-4453-83e1-45307458c7a0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:29.594461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efbab4-91e5-4453-83e1-45307458c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:29.594461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efbab4-91e5-4453-83e1-45307458c7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:29.614770 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-29efbab4-91e5-4453-83e1-45307458c7a0 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:29.615978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-29efbab4-91e5-4453-83e1-45307458c7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:29.673981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1418/2841] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:29.683531 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:29.684975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:29.746922 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37d024-47b8-4684-a714-cf342770c2e0 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:29.750731 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d1fcd-c3d6-4c51-9d30-f42a03b72697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:29.751830 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d1fcd-c3d6-4c51-9d30-f42a03b72697 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:29.752104 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d1fcd-c3d6-4c51-9d30-f42a03b72697 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:29.752369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d1fcd-c3d6-4c51-9d30-f42a03b72697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:29.761295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d1fcd-c3d6-4c51-9d30-f42a03b72697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:29.763044 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d1fcd-c3d6-4c51-9d30-f42a03b72697 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:29.773869 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1424/2842] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:30.047720 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efbab4-91e5-4453-83e1-45307458c7a0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:30.049184 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-29efbab4-91e5-4453-83e1-45307458c7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:30.060749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efbab4-91e5-4453-83e1-45307458c7a0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:30.065736 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:30.066363 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:30.067377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:30.067660 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] PATH_INFO: `/v3/projects/f9b53bb2395c4f8abb2a038464a17d3b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:30.069012 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b53bb2395c4f8abb2a038464a17d3b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:30.070920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:30.120182 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b53bb2395c4f8abb2a038464a17d3b', 'request_id': 'req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.128127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b53bb2395c4f8abb2a038464a17d3b', 'request_id': 'req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.134667 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b53bb2395c4f8abb2a038464a17d3b', 'request_id': 'req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.148132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] The token cache is being invalidate because project f9b53bb2395c4f8abb2a038464a17d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:30.148732 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.149751 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.152691 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1419/2843] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:30 2022] DELETE /identity/v3/projects/f9b53bb2395c4f8abb2a038464a17d3b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:30.158962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d1fcd-c3d6-4c51-9d30-f42a03b72697 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:30.168786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d1fcd-c3d6-4c51-9d30-f42a03b72697 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:30.187549 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:30.227640 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:30.228269 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:30.228550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:30.228854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] PATH_INFO: `/v3/users/5f941368f55947b0a2c83a76a9393ad0` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:30.229984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f941368f55947b0a2c83a76a9393ad0)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:30.231926 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:30.287268 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f941368f55947b0a2c83a76a9393ad0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.294491 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f941368f55947b0a2c83a76a9393ad0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.295192 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f941368f55947b0a2c83a76a9393ad0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.307880 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f941368f55947b0a2c83a76a9393ad0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.308535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-f8261f92-83f2-4f7e-bf07-6740a3b79034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f941368f55947b0a2c83a76a9393ad0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.322231 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1425/2844] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:30 2022] DELETE /identity/v3/users/5f941368f55947b0a2c83a76a9393ad0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:30.588102 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:30.589688 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:30.606760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f5a0e-5a2d-4bc2-af71-28ee8ea4540e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:30.666140 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:30.666760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:30.667035 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:30.667303 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] PATH_INFO: `/v3/projects/240bc122474e4a198bdd32a14d1d0e5e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:30.668399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240bc122474e4a198bdd32a14d1d0e5e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:30.670329 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:30.701169 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:30.718439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240bc122474e4a198bdd32a14d1d0e5e', 'request_id': 'req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.726433 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240bc122474e4a198bdd32a14d1d0e5e', 'request_id': 'req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.732624 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240bc122474e4a198bdd32a14d1d0e5e', 'request_id': 'req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.743785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] The token cache is being invalidate because project 240bc122474e4a198bdd32a14d1d0e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:30.744074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.744746 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8bc77301-ddc1-4e5f-a7fe-fa8ecc0ca984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:30.746231 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1420/2845] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:30 2022] DELETE /identity/v3/projects/240bc122474e4a198bdd32a14d1d0e5e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:30.751831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32a600-a10a-475b-aaf8-b423a94445f0 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:30.752343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32a600-a10a-475b-aaf8-b423a94445f0 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:30.752463 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32a600-a10a-475b-aaf8-b423a94445f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:30.773726 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fe32a600-a10a-475b-aaf8-b423a94445f0 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:30.774941 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fe32a600-a10a-475b-aaf8-b423a94445f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:30.836724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1426/2846] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:30.845797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdad3e9-7bb5-4925-8d31-8687cc28acc8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:30.846084 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdad3e9-7bb5-4925-8d31-8687cc28acc8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:30.846356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdad3e9-7bb5-4925-8d31-8687cc28acc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:30.867534 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7cdad3e9-7bb5-4925-8d31-8687cc28acc8 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:30.868774 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7cdad3e9-7bb5-4925-8d31-8687cc28acc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:30.877442 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1421/2847] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:30.881813 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32a600-a10a-475b-aaf8-b423a94445f0 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:30.891694 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32a600-a10a-475b-aaf8-b423a94445f0 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:30.896296 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:30.896920 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:30.897196 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:30.897462 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:30.898102 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:30.898402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:30.899987 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:30.912329 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1427/2848] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:30.916709 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdad3e9-7bb5-4925-8d31-8687cc28acc8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:30.926737 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdad3e9-7bb5-4925-8d31-8687cc28acc8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:30.931383 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48a09b-ad4f-4897-8fd6-b4a0f895da11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:30.931981 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48a09b-ad4f-4897-8fd6-b4a0f895da11 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:30.932258 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48a09b-ad4f-4897-8fd6-b4a0f895da11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:30.932530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48a09b-ad4f-4897-8fd6-b4a0f895da11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:30.933195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48a09b-ad4f-4897-8fd6-b4a0f895da11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:30.935136 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48a09b-ad4f-4897-8fd6-b4a0f895da11 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:30.962186 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1422/2849] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:30 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:28:30.966312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:30.976984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476856c-7719-4812-b273-cb65d3654aee admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.002766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:31.037184 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e55b7-106a-4f84-a670-2e528fe4bc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.037786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e55b7-106a-4f84-a670-2e528fe4bc11 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.038085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e55b7-106a-4f84-a670-2e528fe4bc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.038353 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e55b7-106a-4f84-a670-2e528fe4bc11 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.038999 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e55b7-106a-4f84-a670-2e528fe4bc11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.040937 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e55b7-106a-4f84-a670-2e528fe4bc11 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.062226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1428/2850] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:30 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:31.066889 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48a09b-ad4f-4897-8fd6-b4a0f895da11 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.077204 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48a09b-ad4f-4897-8fd6-b4a0f895da11 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.082399 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790300cf-7475-49ec-a715-b0888774d510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.082995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790300cf-7475-49ec-a715-b0888774d510 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.083279 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790300cf-7475-49ec-a715-b0888774d510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.083552 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790300cf-7475-49ec-a715-b0888774d510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.084197 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790300cf-7475-49ec-a715-b0888774d510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.085826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790300cf-7475-49ec-a715-b0888774d510 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.096320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1423/2851] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:31.100404 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e55b7-106a-4f84-a670-2e528fe4bc11 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.111595 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e55b7-106a-4f84-a670-2e528fe4bc11 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.116233 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3dee5-4f0d-49c1-afab-487ec8c9df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.116849 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3dee5-4f0d-49c1-afab-487ec8c9df4d admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.117130 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3dee5-4f0d-49c1-afab-487ec8c9df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.117402 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3dee5-4f0d-49c1-afab-487ec8c9df4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.118040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3dee5-4f0d-49c1-afab-487ec8c9df4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.119644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3dee5-4f0d-49c1-afab-487ec8c9df4d admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.131832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1429/2852] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:31.136056 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790300cf-7475-49ec-a715-b0888774d510 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.146773 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790300cf-7475-49ec-a715-b0888774d510 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.151592 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893e556-f9b1-460b-998c-5cb866e09b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.152266 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893e556-f9b1-460b-998c-5cb866e09b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.152549 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893e556-f9b1-460b-998c-5cb866e09b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.152854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893e556-f9b1-460b-998c-5cb866e09b0c admin admin] PATH_INFO: `/v3/projects/eddc6c0a62bf4240bd6a7bad73b1d8d2/users/5a769a4731064ba2b409edf5a124bdef/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.167935 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893e556-f9b1-460b-998c-5cb866e09b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddc6c0a62bf4240bd6a7bad73b1d8d2, user_id=5a769a4731064ba2b409edf5a124bdef, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.170605 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893e556-f9b1-460b-998c-5cb866e09b0c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.184253 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1424/2853] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:31 2022] PUT /identity/v3/projects/eddc6c0a62bf4240bd6a7bad73b1d8d2/users/5a769a4731064ba2b409edf5a124bdef/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:31.195285 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b678104-01f3-42c9-bca8-953f31611ea6 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.195574 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b678104-01f3-42c9-bca8-953f31611ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.195847 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b678104-01f3-42c9-bca8-953f31611ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.223109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-6b678104-01f3-42c9-bca8-953f31611ea6 None None] MFA Rules not processed for user `5a769a4731064ba2b409edf5a124bdef`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:31.224325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-6b678104-01f3-42c9-bca8-953f31611ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.293968 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1430/2854] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:31.298454 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893e556-f9b1-460b-998c-5cb866e09b0c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.358564 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893e556-f9b1-460b-998c-5cb866e09b0c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.363545 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ab814-b230-4cf2-a43f-e1db2c079aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.364029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab814-b230-4cf2-a43f-e1db2c079aed admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.364307 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab814-b230-4cf2-a43f-e1db2c079aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.364591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab814-b230-4cf2-a43f-e1db2c079aed admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.365225 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ab814-b230-4cf2-a43f-e1db2c079aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.367112 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ab814-b230-4cf2-a43f-e1db2c079aed admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.394075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1425/2855] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:31 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:31.398235 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b678104-01f3-42c9-bca8-953f31611ea6 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.410062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b678104-01f3-42c9-bca8-953f31611ea6 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.465163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8b35e-b48e-4e84-8d30-d5fda57222b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.465754 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8b35e-b48e-4e84-8d30-d5fda57222b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.466029 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8b35e-b48e-4e84-8d30-d5fda57222b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.466297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8b35e-b48e-4e84-8d30-d5fda57222b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.466924 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8b35e-b48e-4e84-8d30-d5fda57222b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.468976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8b35e-b48e-4e84-8d30-d5fda57222b8 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.489765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1431/2856] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:31 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:31.493964 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ab814-b230-4cf2-a43f-e1db2c079aed admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.505764 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ab814-b230-4cf2-a43f-e1db2c079aed admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.510362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69296132-a8c8-4780-b9ee-8ff27f52a8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.510965 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69296132-a8c8-4780-b9ee-8ff27f52a8fa admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.511244 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69296132-a8c8-4780-b9ee-8ff27f52a8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.511511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69296132-a8c8-4780-b9ee-8ff27f52a8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.512249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69296132-a8c8-4780-b9ee-8ff27f52a8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.513897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69296132-a8c8-4780-b9ee-8ff27f52a8fa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.525214 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1426/2857] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:31.529708 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8b35e-b48e-4e84-8d30-d5fda57222b8 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.539766 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8b35e-b48e-4e84-8d30-d5fda57222b8 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.544352 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c59a1-d106-43a4-8633-d17249c4a976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.545023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c59a1-d106-43a4-8633-d17249c4a976 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.545302 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c59a1-d106-43a4-8633-d17249c4a976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.545573 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c59a1-d106-43a4-8633-d17249c4a976 admin admin] PATH_INFO: `/v3/projects/3b4df42e770548fea2c5069952a53c8d/users/043f5b3b127e4eaba139c49f70789086/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.560659 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c59a1-d106-43a4-8633-d17249c4a976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4df42e770548fea2c5069952a53c8d, user_id=043f5b3b127e4eaba139c49f70789086, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.564056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c59a1-d106-43a4-8633-d17249c4a976 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.576851 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1432/2858] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:31 2022] PUT /identity/v3/projects/3b4df42e770548fea2c5069952a53c8d/users/043f5b3b127e4eaba139c49f70789086/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:31.581830 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69296132-a8c8-4780-b9ee-8ff27f52a8fa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.591632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69296132-a8c8-4780-b9ee-8ff27f52a8fa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.612888 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:31.652378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b0310-d526-430e-8166-abf381562bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.653016 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b0310-d526-430e-8166-abf381562bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.653299 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b0310-d526-430e-8166-abf381562bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.653569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b0310-d526-430e-8166-abf381562bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.654584 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b0310-d526-430e-8166-abf381562bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.656177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b0310-d526-430e-8166-abf381562bbd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.665902 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1427/2859] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:31.670104 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c59a1-d106-43a4-8633-d17249c4a976 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.675630 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:31.681707 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c59a1-d106-43a4-8633-d17249c4a976 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.686650 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.687309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.687576 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.688343 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] PATH_INFO: `/v3/projects/3b4df42e770548fea2c5069952a53c8d/users/043f5b3b127e4eaba139c49f70789086/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.690317 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4df42e770548fea2c5069952a53c8d, user_id=043f5b3b127e4eaba139c49f70789086, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.692967 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.703023 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1433/2860] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:31 2022] PUT /identity/v3/projects/3b4df42e770548fea2c5069952a53c8d/users/043f5b3b127e4eaba139c49f70789086/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:31.715851 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86652-aec8-43dd-9e8d-c2d9a9460928 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.716145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86652-aec8-43dd-9e8d-c2d9a9460928 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.716439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86652-aec8-43dd-9e8d-c2d9a9460928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.745416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-efc86652-aec8-43dd-9e8d-c2d9a9460928 None None] MFA Rules not processed for user `043f5b3b127e4eaba139c49f70789086`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:31.746630 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-efc86652-aec8-43dd-9e8d-c2d9a9460928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.816000 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1428/2861] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:31.827155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.829246 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.886905 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d1d42-deb7-4e20-a718-649a77408a69 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.890644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.891639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.891639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.891902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.900736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.902187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.911906 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1434/2862] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:31.956687 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc86652-aec8-43dd-9e8d-c2d9a9460928 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.958189 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-efc86652-aec8-43dd-9e8d-c2d9a9460928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.970341 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc86652-aec8-43dd-9e8d-c2d9a9460928 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.974022 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434de249-afb9-4faf-8c4f-25a390805cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.974617 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434de249-afb9-4faf-8c4f-25a390805cea service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.974899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434de249-afb9-4faf-8c4f-25a390805cea service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.975172 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434de249-afb9-4faf-8c4f-25a390805cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:32.033033 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434de249-afb9-4faf-8c4f-25a390805cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:32.034474 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434de249-afb9-4faf-8c4f-25a390805cea service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:32.043360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1429/2863] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:32.691477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:32.692883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:32.751430 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c9eda-e252-4763-a6ce-c90dba7aa299 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:32.754975 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772145a-998c-42e4-90b9-129a58d98be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:32.755571 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772145a-998c-42e4-90b9-129a58d98be1 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:32.755845 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772145a-998c-42e4-90b9-129a58d98be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:32.756109 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772145a-998c-42e4-90b9-129a58d98be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:32.757857 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7772145a-998c-42e4-90b9-129a58d98be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:32.768483 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772145a-998c-42e4-90b9-129a58d98be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:32.769947 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772145a-998c-42e4-90b9-129a58d98be1 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:32.777808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1435/2864] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:28:35.432514 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434de249-afb9-4faf-8c4f-25a390805cea service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:35.478989 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:35.496145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434de249-afb9-4faf-8c4f-25a390805cea service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:35.501427 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:35.502089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:35.502370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:35.502642 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] PATH_INFO: `/v3/users/5a769a4731064ba2b409edf5a124bdef` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:35.503940 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a769a4731064ba2b409edf5a124bdef)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:35.505975 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:35.553371 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a769a4731064ba2b409edf5a124bdef'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:35.566459 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a769a4731064ba2b409edf5a124bdef'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:35.567166 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a769a4731064ba2b409edf5a124bdef'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:35.567822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a769a4731064ba2b409edf5a124bdef'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:35.575403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a769a4731064ba2b409edf5a124bdef'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:35.588330 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1430/2865] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:35 2022] DELETE /identity/v3/users/5a769a4731064ba2b409edf5a124bdef => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:35.593146 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2dedd-8018-4ff0-991d-148b12514991 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:35.593642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2dedd-8018-4ff0-991d-148b12514991 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:35.593746 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2dedd-8018-4ff0-991d-148b12514991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:35.614051 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-01b2dedd-8018-4ff0-991d-148b12514991 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:35.615206 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-01b2dedd-8018-4ff0-991d-148b12514991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:35.676306 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1436/2866] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:35.685985 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:35.687393 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:35.744221 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeb146-3b93-4138-873b-ef66a18140ea admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:35.747887 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b32ff3-0b3a-4fa0-99b7-1c47d65626cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:35.748488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32ff3-0b3a-4fa0-99b7-1c47d65626cc service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:35.748803 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32ff3-0b3a-4fa0-99b7-1c47d65626cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:35.749074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32ff3-0b3a-4fa0-99b7-1c47d65626cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:35.758560 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b32ff3-0b3a-4fa0-99b7-1c47d65626cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:35.760004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b32ff3-0b3a-4fa0-99b7-1c47d65626cc service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:35.770841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1431/2867] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:36.053962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2dedd-8018-4ff0-991d-148b12514991 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:36.055561 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-01b2dedd-8018-4ff0-991d-148b12514991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:36.068943 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2dedd-8018-4ff0-991d-148b12514991 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:36.073935 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:36.074569 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:36.074855 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:36.075179 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] PATH_INFO: `/v3/projects/eddc6c0a62bf4240bd6a7bad73b1d8d2` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:36.076449 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eddc6c0a62bf4240bd6a7bad73b1d8d2)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:36.078495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:36.114253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:36.128677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eddc6c0a62bf4240bd6a7bad73b1d8d2', 'request_id': 'req-1b9ad473-ee32-4c22-a87f-088739b7d876'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.134756 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eddc6c0a62bf4240bd6a7bad73b1d8d2', 'request_id': 'req-1b9ad473-ee32-4c22-a87f-088739b7d876'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.144025 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eddc6c0a62bf4240bd6a7bad73b1d8d2', 'request_id': 'req-1b9ad473-ee32-4c22-a87f-088739b7d876'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.158579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] The token cache is being invalidate because project eddc6c0a62bf4240bd6a7bad73b1d8d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:36.158872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.159492 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.161037 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1437/2868] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:36 2022] DELETE /identity/v3/projects/eddc6c0a62bf4240bd6a7bad73b1d8d2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:36.165236 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b32ff3-0b3a-4fa0-99b7-1c47d65626cc service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:36.176704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b32ff3-0b3a-4fa0-99b7-1c47d65626cc service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:36.233017 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:36.233640 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:36.233920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:36.234190 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] PATH_INFO: `/v3/users/043f5b3b127e4eaba139c49f70789086` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:36.236856 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=043f5b3b127e4eaba139c49f70789086)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:36.238897 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:36.244320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:36.286354 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043f5b3b127e4eaba139c49f70789086'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.299096 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043f5b3b127e4eaba139c49f70789086'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.299785 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043f5b3b127e4eaba139c49f70789086'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.300403 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043f5b3b127e4eaba139c49f70789086'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.308377 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a16030f9-3669-4069-9fc3-e5a77d8cd1dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043f5b3b127e4eaba139c49f70789086'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.320331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1432/2869] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:36 2022] DELETE /identity/v3/users/043f5b3b127e4eaba139c49f70789086 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:36.603380 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:36.604991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:36.619201 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ad473-ee32-4c22-a87f-088739b7d876 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:36.675895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:36.676443 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:36.676750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:36.677046 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] PATH_INFO: `/v3/projects/3b4df42e770548fea2c5069952a53c8d` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:36.678147 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4df42e770548fea2c5069952a53c8d)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:36.680032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:36.727864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4df42e770548fea2c5069952a53c8d', 'request_id': 'req-26caf1ad-7e74-4d26-b408-feb79edda9c0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.733907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4df42e770548fea2c5069952a53c8d', 'request_id': 'req-26caf1ad-7e74-4d26-b408-feb79edda9c0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.739418 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4df42e770548fea2c5069952a53c8d', 'request_id': 'req-26caf1ad-7e74-4d26-b408-feb79edda9c0'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.752056 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] The token cache is being invalidate because project 3b4df42e770548fea2c5069952a53c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:36.752345 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.752977 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-26caf1ad-7e74-4d26-b408-feb79edda9c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:36.754439 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1438/2870] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:36 2022] DELETE /identity/v3/projects/3b4df42e770548fea2c5069952a53c8d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:36.759335 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120c23d-7bf7-457b-9afb-b2ddaffa0a47 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:36.759820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120c23d-7bf7-457b-9afb-b2ddaffa0a47 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:36.759929 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120c23d-7bf7-457b-9afb-b2ddaffa0a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:36.781993 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b120c23d-7bf7-457b-9afb-b2ddaffa0a47 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:36.783086 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b120c23d-7bf7-457b-9afb-b2ddaffa0a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:36.841220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1433/2871] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:36.849190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4eb94-aa4d-48eb-9ab0-7c45db78bda2 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:36.849479 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4eb94-aa4d-48eb-9ab0-7c45db78bda2 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:36.849779 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4eb94-aa4d-48eb-9ab0-7c45db78bda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:36.869121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-dff4eb94-aa4d-48eb-9ab0-7c45db78bda2 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:36.870337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-dff4eb94-aa4d-48eb-9ab0-7c45db78bda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:36.879774 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1439/2872] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:36.884540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120c23d-7bf7-457b-9afb-b2ddaffa0a47 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:36.895370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120c23d-7bf7-457b-9afb-b2ddaffa0a47 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:36.900005 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:36.900637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:36.900920 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:36.901192 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:36.901837 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:36.902138 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:36.903722 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:36.916649 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1434/2873] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:36.921253 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4eb94-aa4d-48eb-9ab0-7c45db78bda2 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:36.930563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4eb94-aa4d-48eb-9ab0-7c45db78bda2 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:36.935026 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0661da2-9dc5-4ec8-8396-fe22bedd569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:36.935613 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661da2-9dc5-4ec8-8396-fe22bedd569a admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:36.935886 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661da2-9dc5-4ec8-8396-fe22bedd569a admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:36.936152 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661da2-9dc5-4ec8-8396-fe22bedd569a admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:36.936798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0661da2-9dc5-4ec8-8396-fe22bedd569a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:36.939216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0661da2-9dc5-4ec8-8396-fe22bedd569a admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:36.966550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1440/2874] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:36 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:36.970945 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:36.981132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c169fb-6ee3-4e24-a6cc-d662f87da6dc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.038821 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3279c1b-0ddb-415f-adff-1692c9d5fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.039418 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3279c1b-0ddb-415f-adff-1692c9d5fbc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.039698 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3279c1b-0ddb-415f-adff-1692c9d5fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.039995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3279c1b-0ddb-415f-adff-1692c9d5fbc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.040643 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3279c1b-0ddb-415f-adff-1692c9d5fbc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.042566 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3279c1b-0ddb-415f-adff-1692c9d5fbc0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.064777 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1435/2875] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:36 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:37.069107 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0661da2-9dc5-4ec8-8396-fe22bedd569a admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.079966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0661da2-9dc5-4ec8-8396-fe22bedd569a admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.084615 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c898fa-95a1-493b-91e8-57ccb03700cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.085211 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c898fa-95a1-493b-91e8-57ccb03700cc admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.085576 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c898fa-95a1-493b-91e8-57ccb03700cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.085817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c898fa-95a1-493b-91e8-57ccb03700cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.086602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c898fa-95a1-493b-91e8-57ccb03700cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.088356 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c898fa-95a1-493b-91e8-57ccb03700cc admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.098435 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1441/2876] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:37.102389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3279c1b-0ddb-415f-adff-1692c9d5fbc0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.107565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:37.114978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3279c1b-0ddb-415f-adff-1692c9d5fbc0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.120683 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91e63f-f462-4ebe-8c47-cf12cd91721e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.121288 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91e63f-f462-4ebe-8c47-cf12cd91721e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.121568 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91e63f-f462-4ebe-8c47-cf12cd91721e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.121841 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91e63f-f462-4ebe-8c47-cf12cd91721e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.122482 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91e63f-f462-4ebe-8c47-cf12cd91721e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.124092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91e63f-f462-4ebe-8c47-cf12cd91721e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.135755 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1436/2877] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:37.140309 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c898fa-95a1-493b-91e8-57ccb03700cc admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.151392 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c898fa-95a1-493b-91e8-57ccb03700cc admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.157117 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae01f4-61e7-456b-b8c7-042bde37b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.157915 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae01f4-61e7-456b-b8c7-042bde37b2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.158198 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae01f4-61e7-456b-b8c7-042bde37b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.158476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae01f4-61e7-456b-b8c7-042bde37b2c4 admin admin] PATH_INFO: `/v3/projects/420c1d5e16584d69a2f9aed06dd770bd/users/f4ab67ac2b0b4977a155ec9f535b0d20/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.161594 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:37.174178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae01f4-61e7-456b-b8c7-042bde37b2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=420c1d5e16584d69a2f9aed06dd770bd, user_id=f4ab67ac2b0b4977a155ec9f535b0d20, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.177088 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae01f4-61e7-456b-b8c7-042bde37b2c4 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.189628 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1442/2878] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:37 2022] PUT /identity/v3/projects/420c1d5e16584d69a2f9aed06dd770bd/users/f4ab67ac2b0b4977a155ec9f535b0d20/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:37.255704 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd851056-d922-4942-a197-682f662dbe69 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.255973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd851056-d922-4942-a197-682f662dbe69 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.256247 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd851056-d922-4942-a197-682f662dbe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.286372 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dd851056-d922-4942-a197-682f662dbe69 None None] MFA Rules not processed for user `f4ab67ac2b0b4977a155ec9f535b0d20`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:37.287576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dd851056-d922-4942-a197-682f662dbe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:37.351416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1437/2879] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:37.356007 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae01f4-61e7-456b-b8c7-042bde37b2c4 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.417459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae01f4-61e7-456b-b8c7-042bde37b2c4 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.422557 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12162a7-c0b4-44cd-96d7-bd73876ed807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.423150 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12162a7-c0b4-44cd-96d7-bd73876ed807 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.423432 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12162a7-c0b4-44cd-96d7-bd73876ed807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.423706 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12162a7-c0b4-44cd-96d7-bd73876ed807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.424339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12162a7-c0b4-44cd-96d7-bd73876ed807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.426362 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12162a7-c0b4-44cd-96d7-bd73876ed807 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.453156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1443/2880] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:37 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:37.457827 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd851056-d922-4942-a197-682f662dbe69 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.468511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd851056-d922-4942-a197-682f662dbe69 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.525196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb2132-c8c5-464a-9092-4bab7eac6334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.525836 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb2132-c8c5-464a-9092-4bab7eac6334 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.526115 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb2132-c8c5-464a-9092-4bab7eac6334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.526388 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb2132-c8c5-464a-9092-4bab7eac6334 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.527025 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb2132-c8c5-464a-9092-4bab7eac6334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.529230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb2132-c8c5-464a-9092-4bab7eac6334 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.551444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1438/2881] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:37 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:37.555579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12162a7-c0b4-44cd-96d7-bd73876ed807 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.566902 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12162a7-c0b4-44cd-96d7-bd73876ed807 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.571456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a8e57-3629-490e-b943-5f4ec494f691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.572045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a8e57-3629-490e-b943-5f4ec494f691 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.572320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a8e57-3629-490e-b943-5f4ec494f691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.572614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a8e57-3629-490e-b943-5f4ec494f691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.573255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a8e57-3629-490e-b943-5f4ec494f691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.574852 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a8e57-3629-490e-b943-5f4ec494f691 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.585891 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1444/2882] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:37.590019 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb2132-c8c5-464a-9092-4bab7eac6334 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.600436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb2132-c8c5-464a-9092-4bab7eac6334 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.605052 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1c5ff-0d5a-416f-84e5-ea52cce01d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.605693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c5ff-0d5a-416f-84e5-ea52cce01d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.605974 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c5ff-0d5a-416f-84e5-ea52cce01d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.606248 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c5ff-0d5a-416f-84e5-ea52cce01d5e admin admin] PATH_INFO: `/v3/projects/1636ce7ad0fd416c99c1adf028689e3e/users/afae48d9b6d34445bd6cb4e7e8a81358/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.621127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1c5ff-0d5a-416f-84e5-ea52cce01d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1636ce7ad0fd416c99c1adf028689e3e, user_id=afae48d9b6d34445bd6cb4e7e8a81358, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.623873 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1c5ff-0d5a-416f-84e5-ea52cce01d5e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.637700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1439/2883] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:37 2022] PUT /identity/v3/projects/1636ce7ad0fd416c99c1adf028689e3e/users/afae48d9b6d34445bd6cb4e7e8a81358/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:28:37.642693 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a8e57-3629-490e-b943-5f4ec494f691 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.653156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a8e57-3629-490e-b943-5f4ec494f691 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.710787 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee244a-7808-4f65-a64c-5a32629a52e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.711319 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee244a-7808-4f65-a64c-5a32629a52e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.711600 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee244a-7808-4f65-a64c-5a32629a52e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.711877 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee244a-7808-4f65-a64c-5a32629a52e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.712521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee244a-7808-4f65-a64c-5a32629a52e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.714177 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee244a-7808-4f65-a64c-5a32629a52e9 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.724384 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1445/2884] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:37.728659 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1c5ff-0d5a-416f-84e5-ea52cce01d5e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.738783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1c5ff-0d5a-416f-84e5-ea52cce01d5e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.743356 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.743988 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.744264 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.744530 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] PATH_INFO: `/v3/projects/1636ce7ad0fd416c99c1adf028689e3e/users/afae48d9b6d34445bd6cb4e7e8a81358/roles/ece89e7759bc4d7891cb77a70f05974f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.746438 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1636ce7ad0fd416c99c1adf028689e3e, user_id=afae48d9b6d34445bd6cb4e7e8a81358, role_id=ece89e7759bc4d7891cb77a70f05974f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.749087 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.759711 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1440/2885] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:37 2022] PUT /identity/v3/projects/1636ce7ad0fd416c99c1adf028689e3e/users/afae48d9b6d34445bd6cb4e7e8a81358/roles/ece89e7759bc4d7891cb77a70f05974f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:37.771156 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a082e-d461-4a8a-b64b-5a473d5e3312 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.771447 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a082e-d461-4a8a-b64b-5a473d5e3312 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.771724 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a082e-d461-4a8a-b64b-5a473d5e3312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.799962 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-5c1a082e-d461-4a8a-b64b-5a473d5e3312 None None] MFA Rules not processed for user `afae48d9b6d34445bd6cb4e7e8a81358`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:37.801224 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-5c1a082e-d461-4a8a-b64b-5a473d5e3312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:37.848195 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:37.873225 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1446/2886] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:37.886513 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:37.888179 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:37.925032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:37.949410 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a9be6-be96-45da-9f96-87f6b3347d42 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:37.955345 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cef60-cbae-4547-9838-f8f7b3b52ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:37.956275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cef60-cbae-4547-9838-f8f7b3b52ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:37.956818 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cef60-cbae-4547-9838-f8f7b3b52ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:37.957300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cef60-cbae-4547-9838-f8f7b3b52ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:37.966875 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cef60-cbae-4547-9838-f8f7b3b52ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:37.968321 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cef60-cbae-4547-9838-f8f7b3b52ae8 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:37.977899 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1441/2887] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:38.293019 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a082e-d461-4a8a-b64b-5a473d5e3312 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:38.302504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a082e-d461-4a8a-b64b-5a473d5e3312 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:38.307062 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afae48d9b6d34445bd6cb4e7e8a81358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1636ce7ad0fd416c99c1adf028689e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:38.307653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:38.307936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:38.308218 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:38.308913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:38.309216 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:38.310858 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:38.322865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1447/2888] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:38.327092 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cef60-cbae-4547-9838-f8f7b3b52ae8 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:38.337154 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cef60-cbae-4547-9838-f8f7b3b52ae8 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:38.341790 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ade5c-6c29-41cd-b018-86ba9fe20327 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afae48d9b6d34445bd6cb4e7e8a81358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1636ce7ad0fd416c99c1adf028689e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:38.342378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ade5c-6c29-41cd-b018-86ba9fe20327 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:38.342668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ade5c-6c29-41cd-b018-86ba9fe20327 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:38.342942 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ade5c-6c29-41cd-b018-86ba9fe20327 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:38.343583 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ade5c-6c29-41cd-b018-86ba9fe20327 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:38.346357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ade5c-6c29-41cd-b018-86ba9fe20327 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:38.377230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1442/2889] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:38 2022] POST /identity/v3/projects => generated 354 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:38.381536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:38.391414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c110-f073-4830-8b45-1746c35b4d9a tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:38.450875 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afae48d9b6d34445bd6cb4e7e8a81358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1636ce7ad0fd416c99c1adf028689e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:38.451493 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:38.451804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:38.452080 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:38.452769 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:38.453078 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:38.454911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:38.466154 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1448/2890] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:38.504463 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ade5c-6c29-41cd-b018-86ba9fe20327 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:38.514679 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ade5c-6c29-41cd-b018-86ba9fe20327 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:38.519331 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afae48d9b6d34445bd6cb4e7e8a81358', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1636ce7ad0fd416c99c1adf028689e3e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:38.519948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:38.520230 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:38.520501 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] PATH_INFO: `/v3/projects/0aec8e2f900540188bb0c9f1768696da` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:38.521665 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0aec8e2f900540188bb0c9f1768696da)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:38.523567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:38.571257 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aec8e2f900540188bb0c9f1768696da', 'request_id': 'req-5234c981-007a-4f50-bfed-9173fbf75158'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:38.579036 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aec8e2f900540188bb0c9f1768696da', 'request_id': 'req-5234c981-007a-4f50-bfed-9173fbf75158'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:38.585762 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aec8e2f900540188bb0c9f1768696da', 'request_id': 'req-5234c981-007a-4f50-bfed-9173fbf75158'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:38.596782 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] The token cache is being invalidate because project 0aec8e2f900540188bb0c9f1768696da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:38.597068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:38.597710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5234c981-007a-4f50-bfed-9173fbf75158 tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:38.599220 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1443/2891] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:38 2022] DELETE /identity/v3/projects/0aec8e2f900540188bb0c9f1768696da => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:38.796145 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:38.798950 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:38.988288 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b3938-5e12-4173-90ee-fab6f20a3eda tempest-VolumeQuotaClassesTest-767032090 tempest-VolumeQuotaClassesTest-767032090-project-admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.050397 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.050965 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.051242 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.051510 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] PATH_INFO: `/v3/users/f4ab67ac2b0b4977a155ec9f535b0d20` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.052677 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ab67ac2b0b4977a155ec9f535b0d20)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.054598 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.064536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=100 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:39.100497 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ab67ac2b0b4977a155ec9f535b0d20'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.107428 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ab67ac2b0b4977a155ec9f535b0d20'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.108105 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ab67ac2b0b4977a155ec9f535b0d20'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.120171 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ab67ac2b0b4977a155ec9f535b0d20'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.120859 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ab67ac2b0b4977a155ec9f535b0d20'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.133973 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1449/2892] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:38 2022] DELETE /identity/v3/users/f4ab67ac2b0b4977a155ec9f535b0d20 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:28:39.138613 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b96aa-19bf-4566-9e2c-dd2fe66b7cd5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.139107 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b96aa-19bf-4566-9e2c-dd2fe66b7cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.139223 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b96aa-19bf-4566-9e2c-dd2fe66b7cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.159641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fe0b96aa-19bf-4566-9e2c-dd2fe66b7cd5 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:39.160880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fe0b96aa-19bf-4566-9e2c-dd2fe66b7cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:39.217996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1444/2893] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:39.228533 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.230035 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:39.288339 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42053086-dfed-434e-a3d6-9abe3188f421 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.291991 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7a90a-e425-4467-a3d5-4d7330cb8e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.292610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a90a-e425-4467-a3d5-4d7330cb8e89 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.292896 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a90a-e425-4467-a3d5-4d7330cb8e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.293167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7a90a-e425-4467-a3d5-4d7330cb8e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.302180 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7a90a-e425-4467-a3d5-4d7330cb8e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.303558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7a90a-e425-4467-a3d5-4d7330cb8e89 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.314208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1450/2894] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:39.581370 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b96aa-19bf-4566-9e2c-dd2fe66b7cd5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.582893 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fe0b96aa-19bf-4566-9e2c-dd2fe66b7cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:39.597416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b96aa-19bf-4566-9e2c-dd2fe66b7cd5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.602216 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.602908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.603209 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.603483 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] PATH_INFO: `/v3/projects/420c1d5e16584d69a2f9aed06dd770bd` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.604641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=420c1d5e16584d69a2f9aed06dd770bd)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.606651 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.642488 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:39.655948 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420c1d5e16584d69a2f9aed06dd770bd', 'request_id': 'req-7db8ea43-9833-4d4a-a6a8-0649f6157052'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.663580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420c1d5e16584d69a2f9aed06dd770bd', 'request_id': 'req-7db8ea43-9833-4d4a-a6a8-0649f6157052'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.670593 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420c1d5e16584d69a2f9aed06dd770bd', 'request_id': 'req-7db8ea43-9833-4d4a-a6a8-0649f6157052'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.681498 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] The token cache is being invalidate because project 420c1d5e16584d69a2f9aed06dd770bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:39.681786 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.682416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.683882 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1445/2895] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:39 2022] DELETE /identity/v3/projects/420c1d5e16584d69a2f9aed06dd770bd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:39.688015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7a90a-e425-4467-a3d5-4d7330cb8e89 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:39.697913 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7a90a-e425-4467-a3d5-4d7330cb8e89 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:39.753838 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:39.754460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:39.754739 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:39.755008 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] PATH_INFO: `/v3/users/afae48d9b6d34445bd6cb4e7e8a81358` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:39.756123 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afae48d9b6d34445bd6cb4e7e8a81358)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:39.758069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:39.792771 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:39.807208 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afae48d9b6d34445bd6cb4e7e8a81358'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.814247 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afae48d9b6d34445bd6cb4e7e8a81358'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.815096 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afae48d9b6d34445bd6cb4e7e8a81358'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.826934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afae48d9b6d34445bd6cb4e7e8a81358'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.827550 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-47dc2f8c-5712-4b8a-aca6-9ccb83a6c5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afae48d9b6d34445bd6cb4e7e8a81358'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:39.841736 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1451/2896] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:39 2022] DELETE /identity/v3/users/afae48d9b6d34445bd6cb4e7e8a81358 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:40.114394 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:40.116867 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:40.134123 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8ea43-9833-4d4a-a6a8-0649f6157052 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:40.195822 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:40.196540 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.196761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.197050 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] PATH_INFO: `/v3/projects/1636ce7ad0fd416c99c1adf028689e3e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.198174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1636ce7ad0fd416c99c1adf028689e3e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:40.200100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:40.249021 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1636ce7ad0fd416c99c1adf028689e3e', 'request_id': 'req-ceccdcb6-6374-45dd-ad2b-b64bef24269a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:40.257129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1636ce7ad0fd416c99c1adf028689e3e', 'request_id': 'req-ceccdcb6-6374-45dd-ad2b-b64bef24269a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:40.263278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1636ce7ad0fd416c99c1adf028689e3e', 'request_id': 'req-ceccdcb6-6374-45dd-ad2b-b64bef24269a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:40.274320 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] The token cache is being invalidate because project 1636ce7ad0fd416c99c1adf028689e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:40.274603 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:40.275222 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ceccdcb6-6374-45dd-ad2b-b64bef24269a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:40.276749 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1446/2897] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:40 2022] DELETE /identity/v3/projects/1636ce7ad0fd416c99c1adf028689e3e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:40.282986 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae05cb-09b1-41e5-a7f4-03e1c748326c None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.283280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae05cb-09b1-41e5-a7f4-03e1c748326c None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.283558 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae05cb-09b1-41e5-a7f4-03e1c748326c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.303375 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-31ae05cb-09b1-41e5-a7f4-03e1c748326c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:40.304597 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-31ae05cb-09b1-41e5-a7f4-03e1c748326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:40.364032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1452/2898] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:40.372076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc6654-d149-4ba9-b38e-1f02db9a4565 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.372362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc6654-d149-4ba9-b38e-1f02db9a4565 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.372760 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc6654-d149-4ba9-b38e-1f02db9a4565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.394222 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5edc6654-d149-4ba9-b38e-1f02db9a4565 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:40.395440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5edc6654-d149-4ba9-b38e-1f02db9a4565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:40.403275 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1447/2899] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:40.407901 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae05cb-09b1-41e5-a7f4-03e1c748326c None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:40.417518 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae05cb-09b1-41e5-a7f4-03e1c748326c None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:40.422553 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:40.423214 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.423476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.423743 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.424384 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:40.424710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:40.426295 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:40.438626 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1453/2900] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:40.443095 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc6654-d149-4ba9-b38e-1f02db9a4565 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:40.452360 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc6654-d149-4ba9-b38e-1f02db9a4565 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:40.457235 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85ddc-67de-4f7d-bfa5-ac328df00294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:40.457829 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85ddc-67de-4f7d-bfa5-ac328df00294 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.458110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85ddc-67de-4f7d-bfa5-ac328df00294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.458379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f85ddc-67de-4f7d-bfa5-ac328df00294 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.459004 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f85ddc-67de-4f7d-bfa5-ac328df00294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:40.460943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f85ddc-67de-4f7d-bfa5-ac328df00294 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:40.488357 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1448/2901] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:40 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:40.492530 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:40.503155 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef36ef0-46c5-4691-8274-6c5741c98c47 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:40.559952 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b2b2d-180f-482a-9754-5feba4310849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:40.560579 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2b2d-180f-482a-9754-5feba4310849 admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.560867 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2b2d-180f-482a-9754-5feba4310849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.561139 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b2b2d-180f-482a-9754-5feba4310849 admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.561790 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b2b2d-180f-482a-9754-5feba4310849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:40.564552 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b2b2d-180f-482a-9754-5feba4310849 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:40.585238 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1454/2902] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:40 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:40.589439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85ddc-67de-4f7d-bfa5-ac328df00294 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:40.599870 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f85ddc-67de-4f7d-bfa5-ac328df00294 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:40.604467 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a1d5-abbd-4f4e-a032-54c91143a11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:40.605089 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a1d5-abbd-4f4e-a032-54c91143a11c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.605423 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a1d5-abbd-4f4e-a032-54c91143a11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.605668 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a1d5-abbd-4f4e-a032-54c91143a11c admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.606310 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4a1d5-abbd-4f4e-a032-54c91143a11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:40.608384 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4a1d5-abbd-4f4e-a032-54c91143a11c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:40.618544 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1449/2903] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:40.622476 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b2b2d-180f-482a-9754-5feba4310849 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:40.633231 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b2b2d-180f-482a-9754-5feba4310849 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:40.637916 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a936c-24e8-45be-a4c6-9a3c1a62fbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:40.638517 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a936c-24e8-45be-a4c6-9a3c1a62fbdc admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.638799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a936c-24e8-45be-a4c6-9a3c1a62fbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.639069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a936c-24e8-45be-a4c6-9a3c1a62fbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.639713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a936c-24e8-45be-a4c6-9a3c1a62fbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:40.641426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a936c-24e8-45be-a4c6-9a3c1a62fbdc admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:40.651426 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1455/2904] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:40.655565 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a1d5-abbd-4f4e-a032-54c91143a11c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:40.665576 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a1d5-abbd-4f4e-a032-54c91143a11c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:40.670196 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:40.670826 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.671100 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.671366 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] PATH_INFO: `/v3/projects/bd424c3c2b214fb78c6807bec24ecbf0/users/985fa972ab9d4181b96398bd5dac9c3a/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.679990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:40.688340 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd424c3c2b214fb78c6807bec24ecbf0, user_id=985fa972ab9d4181b96398bd5dac9c3a, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:40.691379 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:40.705018 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1450/2905] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:40 2022] PUT /identity/v3/projects/bd424c3c2b214fb78c6807bec24ecbf0/users/985fa972ab9d4181b96398bd5dac9c3a/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:40.716645 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89dad-9eb1-41b1-bf70-5b8d72003813 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.716936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89dad-9eb1-41b1-bf70-5b8d72003813 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.717270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89dad-9eb1-41b1-bf70-5b8d72003813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.748126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-f1a89dad-9eb1-41b1-bf70-5b8d72003813 None None] MFA Rules not processed for user `985fa972ab9d4181b96398bd5dac9c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:40.749505 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-f1a89dad-9eb1-41b1-bf70-5b8d72003813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:40.754414 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:40.822182 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1456/2906] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:40.833060 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:40.834734 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:40.894812 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca12be-2741-4480-8505-e11453b14544 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:40.898444 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841ed5e-b29b-490f-98c7-57d3ff2cc26f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:40.899042 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841ed5e-b29b-490f-98c7-57d3ff2cc26f service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:40.899324 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841ed5e-b29b-490f-98c7-57d3ff2cc26f service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:40.899591 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841ed5e-b29b-490f-98c7-57d3ff2cc26f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:40.908750 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841ed5e-b29b-490f-98c7-57d3ff2cc26f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:40.910402 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841ed5e-b29b-490f-98c7-57d3ff2cc26f service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:40.919428 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1451/2907] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:49.514256 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a89dad-9eb1-41b1-bf70-5b8d72003813 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:49.578748 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a89dad-9eb1-41b1-bf70-5b8d72003813 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:49.583821 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:49.584444 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:49.584765 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:49.585040 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] PATH_INFO: `/v3/users/985fa972ab9d4181b96398bd5dac9c3a` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:49.586163 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=985fa972ab9d4181b96398bd5dac9c3a)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:49.588085 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:49.633110 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985fa972ab9d4181b96398bd5dac9c3a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:49.639872 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985fa972ab9d4181b96398bd5dac9c3a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:49.640546 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985fa972ab9d4181b96398bd5dac9c3a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:49.652907 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985fa972ab9d4181b96398bd5dac9c3a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:49.653536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985fa972ab9d4181b96398bd5dac9c3a'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:49.667257 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1457/2908] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:49 2022] DELETE /identity/v3/users/985fa972ab9d4181b96398bd5dac9c3a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:28:49.671834 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70e9a-edc8-448f-b0c1-ad5226fe9b5c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:49.672327 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70e9a-edc8-448f-b0c1-ad5226fe9b5c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:49.672440 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b70e9a-edc8-448f-b0c1-ad5226fe9b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:49.692323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-94b70e9a-edc8-448f-b0c1-ad5226fe9b5c None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:49.693618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-94b70e9a-edc8-448f-b0c1-ad5226fe9b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:49.752729 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1452/2909] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:49.762709 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:49.764470 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:49.793786 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:49.826795 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07915868-8860-48c0-a6ef-949868fd9aed admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:49.830423 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b75de2-1c50-43ae-bd9f-bf75d78cbbb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:49.831026 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b75de2-1c50-43ae-bd9f-bf75d78cbbb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:49.831337 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b75de2-1c50-43ae-bd9f-bf75d78cbbb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:49.831609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b75de2-1c50-43ae-bd9f-bf75d78cbbb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:49.840459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b75de2-1c50-43ae-bd9f-bf75d78cbbb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:49.841934 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b75de2-1c50-43ae-bd9f-bf75d78cbbb1 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:49.851895 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1458/2910] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:50.118846 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b70e9a-edc8-448f-b0c1-ad5226fe9b5c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.120300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-94b70e9a-edc8-448f-b0c1-ad5226fe9b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:50.135815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b70e9a-edc8-448f-b0c1-ad5226fe9b5c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.142068 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.142776 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.143058 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.143472 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] PATH_INFO: `/v3/projects/bd424c3c2b214fb78c6807bec24ecbf0` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.144983 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd424c3c2b214fb78c6807bec24ecbf0)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.146823 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:50.196249 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd424c3c2b214fb78c6807bec24ecbf0', 'request_id': 'req-ebd1b5fa-89d2-487a-8196-318978a370b2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:50.203767 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd424c3c2b214fb78c6807bec24ecbf0', 'request_id': 'req-ebd1b5fa-89d2-487a-8196-318978a370b2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:50.210759 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd424c3c2b214fb78c6807bec24ecbf0', 'request_id': 'req-ebd1b5fa-89d2-487a-8196-318978a370b2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:50.221074 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] The token cache is being invalidate because project bd424c3c2b214fb78c6807bec24ecbf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:50.221362 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:50.221995 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ebd1b5fa-89d2-487a-8196-318978a370b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:50.223494 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1453/2911] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:50 2022] DELETE /identity/v3/projects/bd424c3c2b214fb78c6807bec24ecbf0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:50.228610 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bdff3-9be9-4370-b66d-72fd70c96e59 None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.228897 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bdff3-9be9-4370-b66d-72fd70c96e59 None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.229167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bdff3-9be9-4370-b66d-72fd70c96e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.248878 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-fa4bdff3-9be9-4370-b66d-72fd70c96e59 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:50.250083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-fa4bdff3-9be9-4370-b66d-72fd70c96e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:50.315169 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1459/2912] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:50.324228 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cfb7-5f00-496b-a614-6df45b7a5c06 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.324507 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cfb7-5f00-496b-a614-6df45b7a5c06 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.324815 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cfb7-5f00-496b-a614-6df45b7a5c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.343962 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3692cfb7-5f00-496b-a614-6df45b7a5c06 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:50.345185 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3692cfb7-5f00-496b-a614-6df45b7a5c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:50.353104 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1454/2913] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:50.357440 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bdff3-9be9-4370-b66d-72fd70c96e59 None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.367710 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bdff3-9be9-4370-b66d-72fd70c96e59 None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.372189 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.372804 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.373075 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.373342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] PATH_INFO: `/v3/domains` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.373978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81592) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:50.374273 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.376043 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:50.387831 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1460/2914] 10.222.0.95 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:50.392148 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cfb7-5f00-496b-a614-6df45b7a5c06 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.401435 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cfb7-5f00-496b-a614-6df45b7a5c06 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.406338 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2498ff-2571-49b2-8a68-8ea05e93c61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.406918 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2498ff-2571-49b2-8a68-8ea05e93c61b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.407193 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2498ff-2571-49b2-8a68-8ea05e93c61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.407457 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2498ff-2571-49b2-8a68-8ea05e93c61b admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.408075 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2498ff-2571-49b2-8a68-8ea05e93c61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.410159 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2498ff-2571-49b2-8a68-8ea05e93c61b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:50.414598 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:50.438796 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1455/2915] 10.222.0.95 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:50 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:50.444069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.458412 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec8558-9f63-45e4-9feb-38e86a2e5020 admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.514047 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56683a-a7f7-4776-8510-2485f10bf5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.514664 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56683a-a7f7-4776-8510-2485f10bf5df admin admin] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.515255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56683a-a7f7-4776-8510-2485f10bf5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.515255 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56683a-a7f7-4776-8510-2485f10bf5df admin admin] PATH_INFO: `/v3/users` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.515812 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56683a-a7f7-4776-8510-2485f10bf5df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.517747 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56683a-a7f7-4776-8510-2485f10bf5df admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:50.539455 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1461/2916] 10.222.0.95 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:50 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:28:50.543548 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2498ff-2571-49b2-8a68-8ea05e93c61b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.554567 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2498ff-2571-49b2-8a68-8ea05e93c61b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.559129 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ffa56-54c0-402e-9914-4eaf7b95147a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.559714 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ffa56-54c0-402e-9914-4eaf7b95147a admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.559990 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ffa56-54c0-402e-9914-4eaf7b95147a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.560277 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ffa56-54c0-402e-9914-4eaf7b95147a admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.560927 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ffa56-54c0-402e-9914-4eaf7b95147a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.562499 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ffa56-54c0-402e-9914-4eaf7b95147a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:50.572381 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1456/2917] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:50.576461 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56683a-a7f7-4776-8510-2485f10bf5df admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.586424 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56683a-a7f7-4776-8510-2485f10bf5df admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.590993 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22684597-31c4-4915-ad5a-6992a26d2c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.591580 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22684597-31c4-4915-ad5a-6992a26d2c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.591864 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22684597-31c4-4915-ad5a-6992a26d2c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.592138 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22684597-31c4-4915-ad5a-6992a26d2c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.592817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22684597-31c4-4915-ad5a-6992a26d2c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.594436 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22684597-31c4-4915-ad5a-6992a26d2c25 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:50.605689 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1462/2918] 10.222.0.95 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:50.609761 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ffa56-54c0-402e-9914-4eaf7b95147a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.620412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ffa56-54c0-402e-9914-4eaf7b95147a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.625032 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.625695 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.625969 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.626252 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] PATH_INFO: `/v3/projects/aa40cfcca369418f81fe9961d1023a31/users/370f2be0de8f409f9a1cb06bb1a6f2d5/roles/c2bcd89a362f4fb3a73e0f8ada320802` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.640470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa40cfcca369418f81fe9961d1023a31, user_id=370f2be0de8f409f9a1cb06bb1a6f2d5, role_id=c2bcd89a362f4fb3a73e0f8ada320802)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.643130 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:50.656106 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1457/2919] 10.222.0.95 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:50 2022] PUT /identity/v3/projects/aa40cfcca369418f81fe9961d1023a31/users/370f2be0de8f409f9a1cb06bb1a6f2d5/roles/c2bcd89a362f4fb3a73e0f8ada320802 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:50.668175 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999566-9d4c-4af4-8f3a-bba4b2fb343d None None] REQUEST_METHOD: `POST` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.668465 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999566-9d4c-4af4-8f3a-bba4b2fb343d None None] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.668899 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999566-9d4c-4af4-8f3a-bba4b2fb343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.697770 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.auth.core [None req-b3999566-9d4c-4af4-8f3a-bba4b2fb343d None None] MFA Rules not processed for user `370f2be0de8f409f9a1cb06bb1a6f2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81592) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:50.698988 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-b3999566-9d4c-4af4-8f3a-bba4b2fb343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:50.718799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:50.766634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1463/2920] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:50.778183 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.780174 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:50.838835 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfb463-0256-4a9e-903a-5dd6ef482452 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.842547 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3079e-b593-4120-bf35-68171a3cc91a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.843158 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3079e-b593-4120-bf35-68171a3cc91a service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.843439 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3079e-b593-4120-bf35-68171a3cc91a service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.843710 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3079e-b593-4120-bf35-68171a3cc91a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.852944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3079e-b593-4120-bf35-68171a3cc91a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.854387 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3079e-b593-4120-bf35-68171a3cc91a service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:50.863265 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1458/2921] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:50.888186 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3999566-9d4c-4af4-8f3a-bba4b2fb343d None None] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:50.949327 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3999566-9d4c-4af4-8f3a-bba4b2fb343d None None] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:50.953914 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:50.954536 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:50.954816 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:50.955084 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] PATH_INFO: `/v3/users/370f2be0de8f409f9a1cb06bb1a6f2d5` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:50.956808 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=370f2be0de8f409f9a1cb06bb1a6f2d5)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:50.958822 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:51.006325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370f2be0de8f409f9a1cb06bb1a6f2d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.012936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370f2be0de8f409f9a1cb06bb1a6f2d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.013602 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370f2be0de8f409f9a1cb06bb1a6f2d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.026776 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370f2be0de8f409f9a1cb06bb1a6f2d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.027603 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370f2be0de8f409f9a1cb06bb1a6f2d5'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.042817 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1464/2922] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:50 2022] DELETE /identity/v3/users/370f2be0de8f409f9a1cb06bb1a6f2d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:51.047177 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7180-625c-4c35-84d9-cc0c0a23c0d1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:51.047465 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7180-625c-4c35-84d9-cc0c0a23c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:51.047742 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7180-625c-4c35-84d9-cc0c0a23c0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:51.068237 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b8cf7180-625c-4c35-84d9-cc0c0a23c0d1 None None] MFA Rules not processed for user `60fc2717779e40a0a55137c62461f753`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:51.069492 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b8cf7180-625c-4c35-84d9-cc0c0a23c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:51.128771 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1459/2923] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:51.139134 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:51.140663 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:51.192167 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:51.205829 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70eb09-7749-4f27-9a2c-54541f6c6cdd admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:51.210699 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:51.211321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:51.211639 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:51.211911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:51.221539 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:51.223083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:51.233197 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1465/2924] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:51.515233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf7180-625c-4c35-84d9-cc0c0a23c0d1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:51.516950 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b8cf7180-625c-4c35-84d9-cc0c0a23c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:51.531038 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf7180-625c-4c35-84d9-cc0c0a23c0d1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:51.536781 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:51.537416 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:51.537700 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:51.538632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] PATH_INFO: `/v3/projects/aa40cfcca369418f81fe9961d1023a31` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:51.539780 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa40cfcca369418f81fe9961d1023a31)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:51.541733 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:51.590637 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:51.591880 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa40cfcca369418f81fe9961d1023a31', 'request_id': 'req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.600824 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa40cfcca369418f81fe9961d1023a31', 'request_id': 'req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.607195 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa40cfcca369418f81fe9961d1023a31', 'request_id': 'req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.623569 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] The token cache is being invalidate because project aa40cfcca369418f81fe9961d1023a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:51.623858 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.624511 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8fec4264-0fb8-4db5-b9f6-04eb749a0f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:51.626020 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1460/2925] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:51 2022] DELETE /identity/v3/projects/aa40cfcca369418f81fe9961d1023a31 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:29:01.228514 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=7 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:01.619541 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:19.873318 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:19.875772 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:20.007226 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd81c-b124-4f3e-a48c-79a3c0495147 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:20.098053 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:20.098567 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:20.099624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:20.099911 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:20.102460 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:20.122750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:20.124583 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:20.137854 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1466/2926] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:29:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:29:21.510009 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46391b3-77b8-4335-94a0-7f652421eb65 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:21.510538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46391b3-77b8-4335-94a0-7f652421eb65 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:21.510538 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46391b3-77b8-4335-94a0-7f652421eb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:21.530283 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b46391b3-77b8-4335-94a0-7f652421eb65 None None] MFA Rules not processed for user `bc1911f5d10c4fe38b6a6c4a0950726a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:29:21.531497 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b46391b3-77b8-4335-94a0-7f652421eb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:21.590753 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1461/2927] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:29:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:29:21.916456 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:21.918003 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:21.977112 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccaba5-776b-45e6-8d3f-05bd290e0402 service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:21.981009 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:21.981609 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:21.981883 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:21.982178 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:21.983918 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:21.993767 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:21.995191 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:22.003714 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1467/2928] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:29:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:29:23.485527 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46391b3-77b8-4335-94a0-7f652421eb65 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:23.486911 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b46391b3-77b8-4335-94a0-7f652421eb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:23.558783 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46391b3-77b8-4335-94a0-7f652421eb65 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:23.563787 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:23.564389 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:23.564703 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:23.564972 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:23.567504 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:23.579368 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:23.580998 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:23.592632 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1462/2929] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:29:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:29:31.999582 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=88 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:33.587914 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=57 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:41.223190 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:41.226614 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:41.249149 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5aae6-f437-4655-8286-cfbb21be73f9 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:41.255535 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:41.256213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:41.256495 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:41.256799 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:41.258590 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:41.272217 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:41.273874 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:41.289258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1468/2930] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:29:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:29:45.158907 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:29:45.172145 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29625b21-2c01-4a6b-a1fb-9c41a4047fcb service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:29:45.178278 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa55a30-e93f-45fe-b518-6a882dbb9ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaab594854824542bd85426b0f16cbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:29:45.178871 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa55a30-e93f-45fe-b518-6a882dbb9ec2 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:29:45.179147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa55a30-e93f-45fe-b518-6a882dbb9ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:29:45.179424 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa55a30-e93f-45fe-b518-6a882dbb9ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:29:45.181165 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dfa55a30-e93f-45fe-b518-6a882dbb9ec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:29:45.194076 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa55a30-e93f-45fe-b518-6a882dbb9ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:29:45.195673 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa55a30-e93f-45fe-b518-6a882dbb9ec2 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:29:45.206626 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1463/2931] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:29:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:29:51.283297 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:29:55.201923 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:31:25.415688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:25.417232 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:25.472069 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c6b45-2058-4d84-ab2e-cc4e50e9c079 service nova] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:25.476688 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '115298313c1243d282d09a8e8005002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:25.477284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:25.477563 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:25.477832 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:25.479541 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:25.490496 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:25.493065 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:25.501930 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1469/2932] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:33.424636 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51adf13-29c8-41ae-bdb9-54a4685a42f5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:33.425187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51adf13-29c8-41ae-bdb9-54a4685a42f5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:33.425187 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51adf13-29c8-41ae-bdb9-54a4685a42f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:33.444602 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f51adf13-29c8-41ae-bdb9-54a4685a42f5 None None] MFA Rules not processed for user `bc1911f5d10c4fe38b6a6c4a0950726a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:31:33.446043 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f51adf13-29c8-41ae-bdb9-54a4685a42f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:33.452925 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1464/2933] 10.222.0.95 () {58 vars in 958 bytes} [Wed Aug 31 19:31:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:31:34.931984 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:34.933504 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:34.948121 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800552-ce0b-4bbe-96fa-493b8b5527b5 service cinder] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:34.952936 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:34.953521 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:34.953798 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:34.954064 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:34.955732 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:34.968685 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:34.970258 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:34.980280 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1470/2934] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:43.439745 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:31:44.975727 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=32 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:24.227028 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51adf13-29c8-41ae-bdb9-54a4685a42f5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:24.238147 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51adf13-29c8-41ae-bdb9-54a4685a42f5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:24.241797 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:24.242555 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:24.242696 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:24.242922 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:24.244943 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:24.260641 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:24.263468 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:24.275291 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1465/2935] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:25.462401 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:25.472800 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aab8ab-0754-41a3-af37-960a7d706c7e service placement] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:25.476433 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:25.477106 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:25.477382 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:25.477653 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:25.479407 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:25.493434 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:25.494865 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:25.507015 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1471/2936] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:31.235900 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:31.238233 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:31.251991 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb4369-a28d-4a30-9bde-71a033f5e094 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:31.256213 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:31.256865 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:31.257132 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:31.257419 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:31.259281 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:31.274839 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:31.276314 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:31.289869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1466/2937] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:33:35.501608 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=6 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:37.940634 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:37.950624 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1c9ad-164a-4e88-afce-b5ffd11b6236 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:37.954270 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f1d5c29df30465ead9834b63c3e93bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:37.954806 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:37.955083 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:37.955369 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:37.957095 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:38.021438 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:38.022888 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:38.033848 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1472/2938] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:41.286323 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=12 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:41.322618 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:41.345971 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50f11-c7b6-4050-b765-c497eab0c008 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:41.351273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:41.351895 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:41.352173 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:41.352663 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:41.355765 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:41.383461 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:41.386430 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:41.399973 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1467/2939] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:44.733744 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:44.745978 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab6ff8-71bf-4e65-a54a-725b4102f367 service neutron] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:44.751223 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:44.751976 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] REQUEST_METHOD: `GET` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:44.752254 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:44.752532 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:44.754299 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:44.766750 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:44.768321 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:44.780312 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1473/2940] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:44.790318 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:44.792645 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:44.808693 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367771ed-69e6-407c-9f18-a6560ddbcb73 service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:44.814127 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:44.814820 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:44.815103 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:44.815378 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] PATH_INFO: `/v3/users/bc1911f5d10c4fe38b6a6c4a0950726a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:44.816508 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc1911f5d10c4fe38b6a6c4a0950726a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:44.818470 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:44.866300 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc1911f5d10c4fe38b6a6c4a0950726a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:44.878978 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc1911f5d10c4fe38b6a6c4a0950726a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:44.879669 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc1911f5d10c4fe38b6a6c4a0950726a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:44.880273 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc1911f5d10c4fe38b6a6c4a0950726a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:44.893285 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc1911f5d10c4fe38b6a6c4a0950726a'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:44.907715 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1468/2941] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:33:44 2022] DELETE /identity/v3/users/bc1911f5d10c4fe38b6a6c4a0950726a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:33:45.082018 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:45.093222 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ab780-cb1c-40dc-9c67-004d8c470dea service placement] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:45.156827 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:45.157459 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:45.157740 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:45.158032 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] PATH_INFO: `/v3/projects/d5bd468566364f12abbacb3928316c78` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:45.159187 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5bd468566364f12abbacb3928316c78)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:45.161119 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:45.207041 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5bd468566364f12abbacb3928316c78', 'request_id': 'req-74c50cce-7806-4a2b-99cf-778452e0be0e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.213089 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5bd468566364f12abbacb3928316c78', 'request_id': 'req-74c50cce-7806-4a2b-99cf-778452e0be0e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.219126 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5bd468566364f12abbacb3928316c78', 'request_id': 'req-74c50cce-7806-4a2b-99cf-778452e0be0e'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.233284 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] The token cache is being invalidate because project d5bd468566364f12abbacb3928316c78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:33:45.233576 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.234213 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.235696 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1474/2942] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:33:45 2022] DELETE /identity/v3/projects/d5bd468566364f12abbacb3928316c78 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:33:45.239682 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:45.241098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:45.255612 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab520d0b-5eb5-404a-83eb-ffeb2aa167b3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:45.310869 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:45.311400 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:45.311677 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:45.311944 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] PATH_INFO: `/v3/users/df02b66f2fbe41448d6c9f85a5e7e4fe` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:45.313097 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=df02b66f2fbe41448d6c9f85a5e7e4fe)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:45.315029 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:45.362996 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df02b66f2fbe41448d6c9f85a5e7e4fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.373886 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:45.376262 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df02b66f2fbe41448d6c9f85a5e7e4fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.377580 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df02b66f2fbe41448d6c9f85a5e7e4fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.378506 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df02b66f2fbe41448d6c9f85a5e7e4fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.392597 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df02b66f2fbe41448d6c9f85a5e7e4fe'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.406098 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1469/2943] 10.222.0.95 () {58 vars in 1224 bytes} [Wed Aug 31 19:33:45 2022] DELETE /identity/v3/users/df02b66f2fbe41448d6c9f85a5e7e4fe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:33:45.579477 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] Authenticating user token {{(pid=81592) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:45.580966 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.fernet_utils [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81592) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:45.595320 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c50cce-7806-4a2b-99cf-778452e0be0e admin admin] Validating token access rules against request {{(pid=81592) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:45.603642 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:45.656325 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60fc2717779e40a0a55137c62461f753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b2aa0a8faf9471ebd699445da59c0dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81592) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:45.657045 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:45.657324 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:45.657589 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] PATH_INFO: `/v3/projects/c12ebc76295b4655924ebad82f1ed832` {{(pid=81592) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:45.658713 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12ebc76295b4655924ebad82f1ed832)` {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:45.660644 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] RBAC: Authorization granted {{(pid=81592) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:45.707161 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12ebc76295b4655924ebad82f1ed832', 'request_id': 'req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.713200 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12ebc76295b4655924ebad82f1ed832', 'request_id': 'req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.719342 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12ebc76295b4655924ebad82f1ed832', 'request_id': 'req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2'} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.731262 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] The token cache is being invalidate because project c12ebc76295b4655924ebad82f1ed832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81592) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:33:45.731548 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.732165 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG keystone.notifications [None req-3d5a7baa-abd1-4718-8f3e-40f00d6288d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81592) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:33:45.733662 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: [pid: 81592|app: 0|req: 1475/2944] 10.222.0.95 () {58 vars in 1233 bytes} [Wed Aug 31 19:33:45 2022] DELETE /identity/v3/projects/c12ebc76295b4655924ebad82f1ed832 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:33:55.401436 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:55.728104 dv-cinder-852210-6-iscsi devstack@keystone.service[81592]: DEBUG dbcounter [-] [81592] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:41.968908 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:41.970921 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:42.124930 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f8c8f-d7ba-47a4-878d-387396fe82aa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:42.184876 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202f75b-a54b-419b-ad6a-2903f171d4e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f8675dcfa4988af3a402b32984b84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5680599cf08e4aecbd6b34fd082197c8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:42.185571 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202f75b-a54b-419b-ad6a-2903f171d4e6 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:42.185845 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202f75b-a54b-419b-ad6a-2903f171d4e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:42.186110 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202f75b-a54b-419b-ad6a-2903f171d4e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:42.187816 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8202f75b-a54b-419b-ad6a-2903f171d4e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:42.198854 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202f75b-a54b-419b-ad6a-2903f171d4e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:42.200412 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202f75b-a54b-419b-ad6a-2903f171d4e6 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:42.211219 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1470/2945] 10.222.0.95 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:52.206510 dv-cinder-852210-6-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}